2024-05-08T06:26:00.986 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T06:26:00.986 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_jcollin@teuthology --name jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi --archive /home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389 --verbose --description fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/mds-flush} -- /home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389/orig.config.yaml 2024-05-08T06:26:01.033 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T06:26:01.116 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389 branch: wip-jcollin-testing-07052024 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/mds-flush} email: jcollin@redhat.com first_in_suite: false job_id: '7697389' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-jcollin-testing-07052024 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: cd693b0b93278c2b128be2d6d93ae4df18c3ba1c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: cd693b0b93278c2b128be2d6d93ae4df18c3ba1c kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: origin-jcollin-testing sha1: 6d0b29a4704de72eab52765a276866448dc1f6a5 owner: scheduled_jcollin@teuthology priority: 2 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5526 sha1: cd693b0b93278c2b128be2d6d93ae4df18c3ba1c sleep_before_teardown: 0 suite: fs:functional suite_branch: origin-jcollin-testing suite_path: /home/teuthworker/src/github.com_joscollin_ceph_6d0b29a4704de72eab52765a276866448dc1f6a5/qa suite_relpath: qa suite_repo: https://github.com/joscollin/ceph.git suite_sha1: 6d0b29a4704de72eab52765a276866448dc1f6a5 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi163.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEjGUnfpYuz1hfmfIk1olfiv3+NVWGgwOJdsPkuNzIFSejmYiKRr2J8ipCci639i/dE2wiRCQvoWj1zoKeuqzoA= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKPBptJFAXy1RPfSH9ism4orrVF6u8zHozwHRD2UWZFoURNJg4XfWnLMB1IkroVAcJHKNX/FdJ7TvKDtZv80fsk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_flush teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_06:15:23 tube: smithi user: jcollin verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T06:26:01.116 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_joscollin_ceph_6d0b29a4704de72eab52765a276866448dc1f6a5/qa; will attempt to use it 2024-05-08T06:26:01.117 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_joscollin_ceph_6d0b29a4704de72eab52765a276866448dc1f6a5/qa/tasks 2024-05-08T06:26:01.117 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T06:26:01.118 INFO:teuthology.task.internal:Checking packages... 2024-05-08T06:26:01.142 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c' 2024-05-08T06:26:01.142 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T06:26:01.142 INFO:teuthology.packaging:ref: None 2024-05-08T06:26:01.142 INFO:teuthology.packaging:tag: None 2024-05-08T06:26:01.142 INFO:teuthology.packaging:branch: wip-jcollin-testing-07052024 2024-05-08T06:26:01.142 INFO:teuthology.packaging:sha1: cd693b0b93278c2b128be2d6d93ae4df18c3ba1c 2024-05-08T06:26:01.143 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-jcollin-testing-07052024 2024-05-08T06:26:01.365 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3538.gcd693b0b 2024-05-08T06:26:01.370 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T06:26:01.376 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T06:26:01.377 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T06:26:01.424 INFO:teuthology.task.internal:Saving configuration 2024-05-08T06:26:01.440 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T06:26:01.446 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T06:26:01.472 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 06:20:10.656817', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEjGUnfpYuz1hfmfIk1olfiv3+NVWGgwOJdsPkuNzIFSejmYiKRr2J8ipCci639i/dE2wiRCQvoWj1zoKeuqzoA='} 2024-05-08T06:26:01.495 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 06:20:10.658358', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKPBptJFAXy1RPfSH9ism4orrVF6u8zHozwHRD2UWZFoURNJg4XfWnLMB1IkroVAcJHKNX/FdJ7TvKDtZv80fsk='} 2024-05-08T06:26:01.495 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T06:26:01.501 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-08T06:26:01.501 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-08T06:26:01.501 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T06:26:01.614 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faf485b6160>, signals=[15]) 2024-05-08T06:26:01.614 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T06:26:01.620 INFO:teuthology.task.internal:Opening connections... 2024-05-08T06:26:01.621 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:26:01.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T06:26:01.700 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T06:26:01.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T06:26:01.774 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T06:26:01.781 DEBUG:teuthology.orchestra.run.smithi163:> uname -m 2024-05-08T06:26:01.799 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2024-05-08T06:26:01.799 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2024-05-08T06:26:01.856 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream" 2024-05-08T06:26:01.856 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="9" 2024-05-08T06:26:01.856 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos" 2024-05-08T06:26:01.856 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora" 2024-05-08T06:26:01.856 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="9" 2024-05-08T06:26:01.856 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el9" 2024-05-08T06:26:01.856 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T06:26:01.856 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2024-05-08T06:26:01.856 INFO:teuthology.orchestra.run.smithi163.stdout:LOGO="fedora-logo-icon" 2024-05-08T06:26:01.856 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T06:26:01.857 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/" 2024-05-08T06:26:01.857 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T06:26:01.857 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T06:26:01.857 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T06:26:01.857 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2024-05-08T06:26:01.880 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-05-08T06:26:01.898 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-05-08T06:26:01.899 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-05-08T06:26:01.955 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-05-08T06:26:01.955 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-05-08T06:26:01.955 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-05-08T06:26:01.956 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-05-08T06:26:01.956 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-05-08T06:26:01.956 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-05-08T06:26:01.956 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T06:26:01.956 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-05-08T06:26:01.956 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-05-08T06:26:01.956 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T06:26:01.956 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-05-08T06:26:01.956 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T06:26:01.956 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T06:26:01.956 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T06:26:01.956 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-05-08T06:26:01.983 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T06:26:01.993 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T06:26:02.063 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T06:26:02.063 DEBUG:teuthology.orchestra.run.smithi163:> test '!' -e /home/ubuntu/cephtest 2024-05-08T06:26:02.066 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-05-08T06:26:02.082 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T06:26:02.107 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T06:26:02.108 DEBUG:teuthology.orchestra.run.smithi163:> test -z $(ls -A /var/lib/ceph) 2024-05-08T06:26:02.122 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-05-08T06:26:02.209 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T06:26:02.265 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T06:26:02.279 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-05-08T06:26:02.280 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-05-08T06:26:02.280 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-05-08T06:26:02.280 DEBUG:teuthology.orchestra.run.smithi163:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T06:26:02.280 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T06:26:02.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:26:02.297 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2024-05-08T06:26:02.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:26:02.298 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-05-08T06:26:02.354 INFO:teuthology.orchestra.run.smithi163.stdout:5.14.0-437.el9.x86_64 2024-05-08T06:26:02.355 INFO:teuthology.task.kernel:Running kernel on smithi163: 5.14.0-437.el9.x86_64 2024-05-08T06:26:02.355 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum install -y kernel 2024-05-08T06:26:02.356 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-437.el9.x86_64 2024-05-08T06:26:02.356 INFO:teuthology.task.kernel:Running kernel on smithi192: 5.14.0-437.el9.x86_64 2024-05-08T06:26:02.356 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum install -y kernel 2024-05-08T06:26:07.066 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.1 MB 00:03 2024-05-08T06:26:07.231 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.1 MB 00:03 2024-05-08T06:26:10.407 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 19 MB 00:02 2024-05-08T06:26:25.057 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-05-08T06:26:29.374 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.2 MB 00:00 2024-05-08T06:26:30.749 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 16 kB 00:00 2024-05-08T06:26:32.629 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-08T06:26:36.196 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - CRB 285 kB/s | 6.2 MB 00:22 2024-05-08T06:26:37.843 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-08T06:26:38.039 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-05-08T06:26:39.841 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T06:26:39.841 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T06:26:39.924 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T06:26:39.931 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T06:26:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-05-08T06:26:39.931 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T06:26:39.931 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T06:26:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T06:26:39.932 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-08T06:26:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T06:26:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T06:26:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T06:26:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:26:39.932 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T06:26:39.932 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-05-08T06:26:39.932 INFO:teuthology.orchestra.run.smithi192.stdout:Install 4 Packages 2024-05-08T06:26:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:26:39.933 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 78 M 2024-05-08T06:26:39.933 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 123 M 2024-05-08T06:26:39.933 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T06:26:40.311 INFO:teuthology.orchestra.run.smithi192.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 5.1 MB/s | 484 kB 00:00 2024-05-08T06:26:40.788 INFO:teuthology.orchestra.run.smithi192.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 27 MB/s | 15 MB 00:00 2024-05-08T06:26:41.361 INFO:teuthology.orchestra.run.smithi192.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 30 MB/s | 34 MB 00:01 2024-05-08T06:26:41.661 INFO:teuthology.orchestra.run.smithi192.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 21 MB/s | 28 MB 00:01 2024-05-08T06:26:41.661 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T06:26:41.662 INFO:teuthology.orchestra.run.smithi192.stdout:Total 45 MB/s | 78 MB 00:01 2024-05-08T06:26:42.051 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 5.5 MB/s | 21 MB 00:03 2024-05-08T06:26:42.349 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T06:26:42.666 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T06:26:42.666 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T06:26:43.571 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T06:26:43.571 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T06:26:45.996 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T06:26:46.609 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T06:26:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T06:26:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-08T06:26:49.221 INFO:teuthology.orchestra.run.smithi163.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T06:26:49.221 INFO:teuthology.orchestra.run.smithi163.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T06:26:49.317 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-05-08T06:26:49.325 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-05-08T06:26:49.325 INFO:teuthology.orchestra.run.smithi163.stdout: Package Architecture Version Repository Size 2024-05-08T06:26:49.325 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-05-08T06:26:49.325 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-05-08T06:26:49.325 INFO:teuthology.orchestra.run.smithi163.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T06:26:49.325 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-05-08T06:26:49.325 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T06:26:49.325 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T06:26:49.325 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T06:26:49.325 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:26:49.325 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-05-08T06:26:49.325 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-05-08T06:26:49.325 INFO:teuthology.orchestra.run.smithi163.stdout:Install 4 Packages 2024-05-08T06:26:49.326 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:26:49.326 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 78 M 2024-05-08T06:26:49.326 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 123 M 2024-05-08T06:26:49.326 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-05-08T06:26:49.369 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T06:26:49.507 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T06:26:49.691 INFO:teuthology.orchestra.run.smithi163.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.0 MB/s | 484 kB 00:00 2024-05-08T06:26:49.991 INFO:teuthology.orchestra.run.smithi163.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 29 MB/s | 15 MB 00:00 2024-05-08T06:26:50.400 INFO:teuthology.orchestra.run.smithi163.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 36 MB/s | 34 MB 00:00 2024-05-08T06:26:50.650 INFO:teuthology.orchestra.run.smithi163.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 29 MB/s | 28 MB 00:00 2024-05-08T06:26:50.651 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-05-08T06:26:50.651 INFO:teuthology.orchestra.run.smithi163.stdout:Total 59 MB/s | 78 MB 00:01 2024-05-08T06:26:51.335 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-05-08T06:26:51.703 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-05-08T06:26:51.703 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-05-08T06:26:52.638 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-05-08T06:26:52.639 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-05-08T06:26:54.223 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T06:26:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T06:26:55.074 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-05-08T06:26:55.801 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T06:26:55.883 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T06:26:58.583 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T06:26:58.727 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T06:26:59.124 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T06:27:03.433 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T06:27:03.538 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T06:27:08.245 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T06:27:28.621 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T06:27:28.660 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T06:27:28.995 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T06:27:28.996 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T06:27:28.996 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T06:27:28.996 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T06:27:29.812 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T06:27:29.812 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:27:29.812 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T06:27:29.813 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T06:27:29.813 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T06:27:29.813 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T06:27:29.813 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T06:27:29.813 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:27:29.813 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T06:27:30.170 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T06:27:30.286 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T06:27:30.286 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T06:27:30.286 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T06:27:30.286 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T06:27:30.286 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T06:27:30.287 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-05-08T06:27:30.287 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi192.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T06:27:30.287 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T06:27:30.336 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T06:27:30.336 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T06:27:30.337 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -qi grub2-tools 2024-05-08T06:27:30.367 INFO:teuthology.orchestra.run.smithi192.stdout:Name : grub2-tools 2024-05-08T06:27:30.367 INFO:teuthology.orchestra.run.smithi192.stdout:Epoch : 1 2024-05-08T06:27:30.367 INFO:teuthology.orchestra.run.smithi192.stdout:Version : 2.06 2024-05-08T06:27:30.367 INFO:teuthology.orchestra.run.smithi192.stdout:Release : 68.el9 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout:Architecture: x86_64 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout:Group : Unspecified 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout:Size : 8264254 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout:License : GPLv3+ 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout:Packager : builder@centos.org 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout:Vendor : CentOS 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout:Summary : Support tools for GRUB. 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout:Description : 2024-05-08T06:27:30.368 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:27:30.369 INFO:teuthology.orchestra.run.smithi192.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T06:27:30.369 INFO:teuthology.orchestra.run.smithi192.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T06:27:30.369 INFO:teuthology.orchestra.run.smithi192.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T06:27:30.369 INFO:teuthology.orchestra.run.smithi192.stdout:hardware devices. 2024-05-08T06:27:30.369 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:27:30.369 INFO:teuthology.orchestra.run.smithi192.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T06:27:30.370 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T06:27:30.370 INFO:teuthology.task.kernel:Updating grub on smithi192 to boot 5.14.0-444.el9.x86_64 2024-05-08T06:27:30.370 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T06:27:31.031 INFO:teuthology.orchestra.run.smithi192.stderr:Generating grub configuration file ... 2024-05-08T06:27:32.413 INFO:teuthology.orchestra.run.smithi192.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T06:27:32.447 INFO:teuthology.orchestra.run.smithi192.stderr:done 2024-05-08T06:27:32.449 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-05-08T06:27:32.465 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.TH3M9212Gf 2024-05-08T06:27:32.466 DEBUG:teuthology.orchestra.run.smithi192:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TH3M9212Gf 2024-05-08T06:27:32.543 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.TH3M9212Gf 2024-05-08T06:27:32.683 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.TH3M9212Gf is 6KB 2024-05-08T06:27:32.731 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.TH3M9212Gf 2024-05-08T06:27:32.745 DEBUG:teuthology.orchestra.run.smithi192:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T06:27:32.815 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T06:27:32.815 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T06:27:32.816 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T06:27:32.816 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T06:27:32.817 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T06:27:32.964 DEBUG:teuthology.orchestra.run.smithi192:> sudo shutdown -r now 2024-05-08T06:27:38.035 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T06:27:38.074 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T06:27:38.467 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T06:27:38.468 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T06:27:38.468 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T06:27:38.468 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T06:27:39.267 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T06:27:39.268 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:27:39.268 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-05-08T06:27:39.268 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T06:27:39.268 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T06:27:39.268 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T06:27:39.268 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T06:27:39.268 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:27:39.268 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-05-08T06:27:39.622 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T06:27:39.686 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T06:27:39.686 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T06:27:39.687 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T06:27:39.687 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T06:27:39.687 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T06:27:39.687 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-08T06:27:39.687 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi163.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T06:27:39.687 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T06:27:39.736 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T06:27:39.736 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T06:27:39.736 DEBUG:teuthology.orchestra.run.smithi163:> sudo rpm -qi grub2-tools 2024-05-08T06:27:39.766 INFO:teuthology.orchestra.run.smithi163.stdout:Name : grub2-tools 2024-05-08T06:27:39.766 INFO:teuthology.orchestra.run.smithi163.stdout:Epoch : 1 2024-05-08T06:27:39.766 INFO:teuthology.orchestra.run.smithi163.stdout:Version : 2.06 2024-05-08T06:27:39.766 INFO:teuthology.orchestra.run.smithi163.stdout:Release : 68.el9 2024-05-08T06:27:39.767 INFO:teuthology.orchestra.run.smithi163.stdout:Architecture: x86_64 2024-05-08T06:27:39.767 INFO:teuthology.orchestra.run.smithi163.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T06:27:39.767 INFO:teuthology.orchestra.run.smithi163.stdout:Group : Unspecified 2024-05-08T06:27:39.767 INFO:teuthology.orchestra.run.smithi163.stdout:Size : 8264254 2024-05-08T06:27:39.767 INFO:teuthology.orchestra.run.smithi163.stdout:License : GPLv3+ 2024-05-08T06:27:39.767 INFO:teuthology.orchestra.run.smithi163.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T06:27:39.767 INFO:teuthology.orchestra.run.smithi163.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T06:27:39.767 INFO:teuthology.orchestra.run.smithi163.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T06:27:39.767 INFO:teuthology.orchestra.run.smithi163.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T06:27:39.767 INFO:teuthology.orchestra.run.smithi163.stdout:Packager : builder@centos.org 2024-05-08T06:27:39.768 INFO:teuthology.orchestra.run.smithi163.stdout:Vendor : CentOS 2024-05-08T06:27:39.768 INFO:teuthology.orchestra.run.smithi163.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T06:27:39.768 INFO:teuthology.orchestra.run.smithi163.stdout:Summary : Support tools for GRUB. 2024-05-08T06:27:39.768 INFO:teuthology.orchestra.run.smithi163.stdout:Description : 2024-05-08T06:27:39.768 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:27:39.768 INFO:teuthology.orchestra.run.smithi163.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T06:27:39.768 INFO:teuthology.orchestra.run.smithi163.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T06:27:39.768 INFO:teuthology.orchestra.run.smithi163.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T06:27:39.769 INFO:teuthology.orchestra.run.smithi163.stdout:hardware devices. 2024-05-08T06:27:39.769 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:27:39.769 INFO:teuthology.orchestra.run.smithi163.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T06:27:39.770 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T06:27:39.770 INFO:teuthology.task.kernel:Updating grub on smithi163 to boot 5.14.0-444.el9.x86_64 2024-05-08T06:27:39.770 DEBUG:teuthology.orchestra.run.smithi163:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T06:27:40.415 INFO:teuthology.orchestra.run.smithi163.stderr:Generating grub configuration file ... 2024-05-08T06:27:41.858 INFO:teuthology.orchestra.run.smithi163.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T06:27:41.881 INFO:teuthology.orchestra.run.smithi163.stderr:done 2024-05-08T06:27:41.883 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2024-05-08T06:27:41.939 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.Nbxa9wrzHb 2024-05-08T06:27:41.939 DEBUG:teuthology.orchestra.run.smithi163:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Nbxa9wrzHb 2024-05-08T06:27:42.016 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.Nbxa9wrzHb 2024-05-08T06:27:42.159 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.Nbxa9wrzHb is 6KB 2024-05-08T06:27:42.207 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.Nbxa9wrzHb 2024-05-08T06:27:42.221 DEBUG:teuthology.orchestra.run.smithi163:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T06:27:42.283 INFO:teuthology.orchestra.run.smithi163.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T06:27:42.283 INFO:teuthology.orchestra.run.smithi163.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T06:27:42.283 INFO:teuthology.orchestra.run.smithi163.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T06:27:42.283 INFO:teuthology.orchestra.run.smithi163.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T06:27:42.284 DEBUG:teuthology.orchestra.run.smithi163:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T06:27:42.432 DEBUG:teuthology.orchestra.run.smithi163:> sudo shutdown -r now 2024-05-08T06:28:02.982 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-08T06:28:02.983 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-08T06:28:02.983 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T06:28:12.441 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-08T06:28:12.441 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-05-08T06:28:12.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T06:28:21.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2024-05-08T06:28:30.414 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-08T06:28:30.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T06:28:30.878 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-05-08T06:28:33.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2024-05-08T06:28:39.886 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-05-08T06:28:39.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T06:28:42.946 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-05-08T06:28:45.474 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-08T06:28:45.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T06:28:54.954 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-05-08T06:28:54.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T06:28:58.014 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-05-08T06:29:13.026 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-05-08T06:29:13.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T06:29:17.278 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-05-08T06:29:17.713 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-08T06:29:17.713 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-444.el9.x86_64"... 2024-05-08T06:29:17.713 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-05-08T06:29:17.728 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-444.el9.x86_64 2024-05-08T06:29:17.728 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T06:29:17.728 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T06:29:17.728 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T06:29:18.730 DEBUG:teuthology.parallel:result is None 2024-05-08T06:29:31.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-05-08T06:29:49.554 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-05-08T06:29:49.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T06:29:49.987 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-05-08T06:29:50.383 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-05-08T06:29:50.383 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-08T06:29:50.384 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2024-05-08T06:29:50.398 INFO:teuthology.orchestra.run.smithi163.stdout:5.14.0-444.el9.x86_64 2024-05-08T06:29:50.399 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T06:29:50.399 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T06:29:50.399 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T06:29:51.400 DEBUG:teuthology.parallel:result is None 2024-05-08T06:29:51.400 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T06:29:51.408 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T06:29:51.408 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T06:29:51.410 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T06:29:51.431 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T06:29:51.459 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T06:29:51.471 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T06:29:51.472 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T06:29:51.474 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T06:29:51.507 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T06:29:51.533 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T06:29:51.534 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T06:29:51.536 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T06:29:51.617 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T06:29:51.636 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T06:29:51.641 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T06:29:51.694 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T06:29:51.695 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T06:29:51.703 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T06:29:51.703 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T06:29:51.705 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T06:29:51.733 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T06:29:51.742 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T06:29:51.743 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T06:29:51.769 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T06:29:51.789 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T06:29:51.862 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T06:29:51.906 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T06:29:51.986 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T06:29:52.021 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:29:52.021 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T06:29:52.083 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T06:29:52.143 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T06:29:52.203 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T06:29:52.259 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T06:29:52.298 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:29:52.298 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T06:29:52.366 DEBUG:teuthology.orchestra.run.smithi163:> sudo service rsyslog restart 2024-05-08T06:29:52.370 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-05-08T06:29:52.420 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T06:29:52.423 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T06:29:52.738 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T06:29:52.803 INFO:teuthology.task.internal:Starting timer... 2024-05-08T06:29:52.803 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T06:29:52.831 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T06:29:52.844 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-08T06:29:52.845 DEBUG:teuthology.orchestra.run.smithi163:> sudo service auditd rotate 2024-05-08T06:29:52.974 INFO:teuthology.orchestra.run.smithi163.stdout:Rotating logs: 2024-05-08T06:29:52.976 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-05-08T06:29:53.043 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-05-08T06:29:53.045 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T06:29:53.045 DEBUG:teuthology.orchestra.run.smithi163:> /usr/sbin/getenforce 2024-05-08T06:29:53.095 INFO:teuthology.orchestra.run.smithi163.stdout:Permissive 2024-05-08T06:29:53.095 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-05-08T06:29:53.121 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-05-08T06:29:53.122 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi163.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-05-08T06:29:53.122 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T06:29:53.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:29:53.157 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T06:29:53.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:29:53.189 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T06:29:53.189 DEBUG:teuthology.orchestra.run.smithi163:> sudo /usr/sbin/setenforce permissive 2024-05-08T06:29:53.557 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-05-08T06:29:53.583 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T06:29:53.592 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T06:29:53.609 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T06:29:53.610 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi163.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T06:33:52.815 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi163.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-05-08T06:33:52.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-05-08T06:33:52.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T06:33:52.895 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-05-08T06:33:52.973 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-05-08T06:33:52.974 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-08T06:33:52.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T06:33:53.050 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-05-08T06:33:53.126 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-08T06:33:53.126 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T06:33:53.139 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T06:33:53.140 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T06:33:53.140 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T06:33:53.142 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T06:33:53.143 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T06:33:53.169 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T06:33:53.184 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T06:33:53.197 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T06:33:53.207 INFO:teuthology.orchestra.run.smithi163.stderr:sudo: ntpd: command not found 2024-05-08T06:33:53.211 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T06:33:53.218 INFO:teuthology.orchestra.run.smithi163.stdout:506 Cannot talk to daemon 2024-05-08T06:33:53.232 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T06:33:53.236 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-05-08T06:33:53.246 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T06:33:53.247 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-05-08T06:33:53.261 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T06:33:53.275 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T06:33:53.299 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-05-08T06:33:53.302 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T06:33:53.303 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-05-08T06:33:53.303 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:33:53.303 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:33:53.303 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:33:53.303 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:33:53.331 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-05-08T06:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T06:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-05-08T06:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:33:53.336 INFO:teuthology.run_tasks:Running task install... 2024-05-08T06:33:53.348 DEBUG:teuthology.task.install:project ceph 2024-05-08T06:33:53.348 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c'}} 2024-05-08T06:33:53.348 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c'} 2024-05-08T06:33:53.348 INFO:teuthology.task.install:Using flavor: default 2024-05-08T06:33:53.355 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T06:33:53.356 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T06:33:53.356 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c', 'tag': None, 'wait_for_package': False} 2024-05-08T06:33:53.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd693b0b93278c2b128be2d6d93ae4df18c3ba1c 2024-05-08T06:33:53.358 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c', 'tag': None, 'wait_for_package': False} 2024-05-08T06:33:53.359 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd693b0b93278c2b128be2d6d93ae4df18c3ba1c 2024-05-08T06:33:53.594 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/ 2024-05-08T06:33:53.595 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gcd693b0b 2024-05-08T06:33:53.598 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/ 2024-05-08T06:33:53.598 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gcd693b0b 2024-05-08T06:33:53.708 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T06:33:53.709 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:33:53.709 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T06:33:53.735 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T06:33:53.735 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T06:33:53.763 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T06:33:53.763 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:33:53.763 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T06:33:53.789 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T06:33:53.790 DEBUG:teuthology.orchestra.run.smithi163:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T06:33:53.801 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T06:33:53.856 DEBUG:teuthology.orchestra.run.smithi163:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T06:33:53.876 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T06:33:53.930 DEBUG:teuthology.orchestra.run.smithi163:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T06:33:53.941 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-05-08T06:33:53.943 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-05-08T06:33:53.995 INFO:teuthology.orchestra.run.smithi163.stdout:check_obsoletes = 1 2024-05-08T06:33:53.996 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum clean all 2024-05-08T06:33:54.173 INFO:teuthology.orchestra.run.smithi192.stdout:54 files removed 2024-05-08T06:33:54.200 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T06:33:54.228 INFO:teuthology.orchestra.run.smithi163.stdout:54 files removed 2024-05-08T06:33:54.255 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T06:33:54.943 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 179 kB/s | 84 kB 00:00 2024-05-08T06:33:54.950 INFO:teuthology.orchestra.run.smithi163.stdout:ceph packages for x86_64 201 kB/s | 84 kB 00:00 2024-05-08T06:33:55.300 INFO:teuthology.orchestra.run.smithi163.stdout:ceph noarch packages 40 kB/s | 12 kB 00:00 2024-05-08T06:33:55.310 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-05-08T06:33:55.935 INFO:teuthology.orchestra.run.smithi163.stdout:ceph source packages 3.2 kB/s | 1.9 kB 00:00 2024-05-08T06:33:55.977 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-05-08T06:33:56.652 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-08T06:33:58.411 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-05-08T06:33:58.778 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.1 MB 00:02 2024-05-08T06:34:00.595 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-05-08T06:34:03.378 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-05-08T06:34:05.146 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-05-08T06:34:05.488 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 6.2 MB 00:01 2024-05-08T06:34:06.206 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-05-08T06:34:07.247 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-05-08T06:34:11.281 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-05-08T06:34:12.198 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 4.4 MB/s | 21 MB 00:04 2024-05-08T06:34:13.642 INFO:teuthology.orchestra.run.smithi163.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T06:34:13.642 INFO:teuthology.orchestra.run.smithi163.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T06:34:13.653 INFO:teuthology.orchestra.run.smithi163.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T06:34:13.654 INFO:teuthology.orchestra.run.smithi163.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T06:34:13.655 INFO:teuthology.orchestra.run.smithi163.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T06:34:13.656 INFO:teuthology.orchestra.run.smithi163.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T06:34:13.657 INFO:teuthology.orchestra.run.smithi163.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T06:34:13.659 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T06:34:13.660 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T06:34:13.721 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-05-08T06:34:13.744 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-05-08T06:34:13.744 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-05-08T06:34:13.744 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-05-08T06:34:13.745 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-05-08T06:34:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T06:34:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 6.6 k 2024-05-08T06:34:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 5.4 M 2024-05-08T06:34:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 891 k 2024-05-08T06:34:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 145 k 2024-05-08T06:34:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 1.5 M 2024-05-08T06:34:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 142 k 2024-05-08T06:34:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 3.5 M 2024-05-08T06:34:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 7.4 M 2024-05-08T06:34:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 49 k 2024-05-08T06:34:13.745 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 11 M 2024-05-08T06:34:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 49 M 2024-05-08T06:34:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 273 k 2024-05-08T06:34:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 764 k 2024-05-08T06:34:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 225 k 2024-05-08T06:34:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-shell x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 27 k 2024-05-08T06:34:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 18 k 2024-05-08T06:34:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T06:34:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T06:34:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T06:34:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 32 k 2024-05-08T06:34:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 746 k 2024-05-08T06:34:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 127 k 2024-05-08T06:34:13.746 INFO:teuthology.orchestra.run.smithi163.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T06:34:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T06:34:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T06:34:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T06:34:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T06:34:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T06:34:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 161 k 2024-05-08T06:34:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 320 k 2024-05-08T06:34:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 301 k 2024-05-08T06:34:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 99 k 2024-05-08T06:34:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 86 k 2024-05-08T06:34:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 3.1 M 2024-05-08T06:34:13.747 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 171 k 2024-05-08T06:34:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T06:34:13.748 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrading: 2024-05-08T06:34:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T06:34:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T06:34:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T06:34:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T06:34:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: librados2 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 3.4 M 2024-05-08T06:34:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 3.1 M 2024-05-08T06:34:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T06:34:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T06:34:13.748 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-05-08T06:34:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T06:34:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T06:34:13.748 INFO:teuthology.orchestra.run.smithi163.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T06:34:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 21 M 2024-05-08T06:34:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 34 k 2024-05-08T06:34:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 2.4 M 2024-05-08T06:34:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 274 k 2024-05-08T06:34:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 4.7 M 2024-05-08T06:34:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 17 M 2024-05-08T06:34:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 16 k 2024-05-08T06:34:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 25 k 2024-05-08T06:34:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T06:34:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T06:34:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T06:34:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T06:34:13.749 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 163 k 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 500 k 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 5.4 M 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T06:34:13.750 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T06:34:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T06:34:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T06:34:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T06:34:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T06:34:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T06:34:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T06:34:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T06:34:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T06:34:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T06:34:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T06:34:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T06:34:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T06:34:13.751 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 45 k 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 135 k 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T06:34:13.752 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T06:34:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T06:34:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T06:34:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T06:34:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T06:34:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T06:34:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T06:34:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T06:34:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T06:34:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T06:34:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T06:34:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T06:34:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T06:34:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T06:34:13.753 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T06:34:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T06:34:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T06:34:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T06:34:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T06:34:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T06:34:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T06:34:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T06:34:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T06:34:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T06:34:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T06:34:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T06:34:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T06:34:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T06:34:13.754 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T06:34:13.755 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout:Installing weak dependencies: 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout:Install 148 Packages 2024-05-08T06:34:13.756 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrade 8 Packages 2024-05-08T06:34:13.757 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:34:13.757 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 226 M 2024-05-08T06:34:13.757 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-05-08T06:34:14.366 INFO:teuthology.orchestra.run.smithi163.stdout:(1/156): ceph-19.0.0-3538.gcd693b0b.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-05-08T06:34:14.791 INFO:teuthology.orchestra.run.smithi163.stdout:(2/156): ceph-fuse-19.0.0-3538.gcd693b0b.el9.x8 2.0 MB/s | 891 kB 00:00 2024-05-08T06:34:15.167 INFO:teuthology.orchestra.run.smithi163.stdout:(3/156): ceph-base-19.0.0-3538.gcd693b0b.el9.x8 5.2 MB/s | 5.4 MB 00:01 2024-05-08T06:34:15.192 INFO:teuthology.orchestra.run.smithi163.stdout:(4/156): ceph-immutable-object-cache-19.0.0-353 361 kB/s | 145 kB 00:00 2024-05-08T06:34:15.400 INFO:teuthology.orchestra.run.smithi163.stdout:(5/156): ceph-mds-19.0.0-3538.gcd693b0b.el9.x86 10 MB/s | 2.4 MB 00:00 2024-05-08T06:34:15.878 INFO:teuthology.orchestra.run.smithi163.stdout:(6/156): ceph-mon-19.0.0-3538.gcd693b0b.el9.x86 9.9 MB/s | 4.7 MB 00:00 2024-05-08T06:34:15.917 INFO:teuthology.orchestra.run.smithi163.stdout:(7/156): ceph-mgr-19.0.0-3538.gcd693b0b.el9.x86 2.1 MB/s | 1.5 MB 00:00 2024-05-08T06:34:16.401 INFO:teuthology.orchestra.run.smithi163.stdout:(8/156): ceph-common-19.0.0-3538.gcd693b0b.el9. 9.3 MB/s | 21 MB 00:02 2024-05-08T06:34:16.493 INFO:teuthology.orchestra.run.smithi163.stdout:(9/156): ceph-selinux-19.0.0-3538.gcd693b0b.el9 275 kB/s | 25 kB 00:00 2024-05-08T06:34:17.310 INFO:teuthology.orchestra.run.smithi163.stdout:(10/156): ceph-osd-19.0.0-3538.gcd693b0b.el9.x8 12 MB/s | 17 MB 00:01 2024-05-08T06:34:17.390 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-05-08T06:34:17.677 INFO:teuthology.orchestra.run.smithi163.stdout:(11/156): cephfs-mirror-19.0.0-3538.gcd693b0b.e 613 kB/s | 225 kB 00:00 2024-05-08T06:34:17.761 INFO:teuthology.orchestra.run.smithi163.stdout:(12/156): cephfs-shell-19.0.0-3538.gcd693b0b.el 325 kB/s | 27 kB 00:00 2024-05-08T06:34:17.836 INFO:teuthology.orchestra.run.smithi163.stdout:(13/156): libcephfs-devel-19.0.0-3538.gcd693b0b 424 kB/s | 32 kB 00:00 2024-05-08T06:34:18.011 INFO:teuthology.orchestra.run.smithi163.stdout:(14/156): libcephfs2-19.0.0-3538.gcd693b0b.el9. 4.2 MB/s | 746 kB 00:00 2024-05-08T06:34:18.153 INFO:teuthology.orchestra.run.smithi163.stdout:(15/156): libcephsqlite-19.0.0-3538.gcd693b0b.e 1.1 MB/s | 163 kB 00:00 2024-05-08T06:34:18.245 INFO:teuthology.orchestra.run.smithi163.stdout:(16/156): librados-devel-19.0.0-3538.gcd693b0b. 1.4 MB/s | 127 kB 00:00 2024-05-08T06:34:18.379 INFO:teuthology.orchestra.run.smithi163.stdout:(17/156): libradosstriper1-19.0.0-3538.gcd693b0 3.7 MB/s | 500 kB 00:00 2024-05-08T06:34:18.846 INFO:teuthology.orchestra.run.smithi163.stdout:(18/156): librgw2-19.0.0-3538.gcd693b0b.el9.x86 11 MB/s | 5.4 MB 00:00 2024-05-08T06:34:18.929 INFO:teuthology.orchestra.run.smithi163.stdout:(19/156): python3-ceph-argparse-19.0.0-3538.gcd 543 kB/s | 45 kB 00:00 2024-05-08T06:34:19.071 INFO:teuthology.orchestra.run.smithi163.stdout:(20/156): python3-ceph-common-19.0.0-3538.gcd69 958 kB/s | 135 kB 00:00 2024-05-08T06:34:19.156 INFO:teuthology.orchestra.run.smithi163.stdout:(21/156): python3-cephfs-19.0.0-3538.gcd693b0b. 1.9 MB/s | 161 kB 00:00 2024-05-08T06:34:19.264 INFO:teuthology.orchestra.run.smithi163.stdout:(22/156): python3-rados-19.0.0-3538.gcd693b0b.e 2.9 MB/s | 320 kB 00:00 2024-05-08T06:34:19.372 INFO:teuthology.orchestra.run.smithi163.stdout:(23/156): python3-rbd-19.0.0-3538.gcd693b0b.el9 2.7 MB/s | 301 kB 00:00 2024-05-08T06:34:19.734 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T06:34:19.735 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T06:34:19.746 INFO:teuthology.orchestra.run.smithi192.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T06:34:19.746 INFO:teuthology.orchestra.run.smithi192.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T06:34:19.747 INFO:teuthology.orchestra.run.smithi192.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T06:34:19.748 INFO:teuthology.orchestra.run.smithi192.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T06:34:19.750 INFO:teuthology.orchestra.run.smithi192.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T06:34:19.752 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T06:34:19.753 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T06:34:19.815 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T06:34:19.832 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-08T06:34:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-08T06:34:19.832 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-08T06:34:19.832 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T06:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T06:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 6.6 k 2024-05-08T06:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 5.4 M 2024-05-08T06:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 891 k 2024-05-08T06:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 145 k 2024-05-08T06:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 1.5 M 2024-05-08T06:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 142 k 2024-05-08T06:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 3.5 M 2024-05-08T06:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 7.4 M 2024-05-08T06:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 49 k 2024-05-08T06:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 11 M 2024-05-08T06:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 49 M 2024-05-08T06:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 273 k 2024-05-08T06:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 764 k 2024-05-08T06:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 225 k 2024-05-08T06:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 27 k 2024-05-08T06:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 18 k 2024-05-08T06:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T06:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T06:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T06:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 32 k 2024-05-08T06:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 746 k 2024-05-08T06:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 127 k 2024-05-08T06:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T06:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T06:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T06:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T06:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T06:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T06:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 161 k 2024-05-08T06:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 320 k 2024-05-08T06:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 301 k 2024-05-08T06:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 99 k 2024-05-08T06:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 86 k 2024-05-08T06:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 3.1 M 2024-05-08T06:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 171 k 2024-05-08T06:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T06:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-05-08T06:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T06:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T06:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T06:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T06:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 3.4 M 2024-05-08T06:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 3.1 M 2024-05-08T06:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T06:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T06:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-08T06:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T06:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T06:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T06:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 21 M 2024-05-08T06:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 34 k 2024-05-08T06:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 2.4 M 2024-05-08T06:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 274 k 2024-05-08T06:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 4.7 M 2024-05-08T06:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 17 M 2024-05-08T06:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 16 k 2024-05-08T06:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 25 k 2024-05-08T06:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T06:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T06:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T06:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T06:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T06:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T06:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T06:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T06:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 163 k 2024-05-08T06:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T06:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T06:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T06:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T06:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T06:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 500 k 2024-05-08T06:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T06:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 5.4 M 2024-05-08T06:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T06:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T06:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T06:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T06:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T06:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T06:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T06:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T06:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T06:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T06:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T06:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T06:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T06:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T06:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T06:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T06:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T06:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 45 k 2024-05-08T06:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 135 k 2024-05-08T06:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T06:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T06:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T06:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T06:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T06:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T06:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T06:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T06:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T06:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T06:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T06:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T06:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T06:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T06:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T06:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T06:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T06:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T06:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T06:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T06:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T06:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T06:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T06:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T06:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T06:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T06:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T06:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T06:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T06:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T06:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T06:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T06:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T06:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T06:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T06:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T06:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T06:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T06:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T06:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T06:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T06:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T06:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T06:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T06:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T06:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T06:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T06:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T06:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T06:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T06:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T06:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T06:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T06:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T06:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T06:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T06:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T06:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T06:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T06:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T06:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T06:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T06:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T06:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-05-08T06:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T06:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T06:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-08T06:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout:Install 148 Packages 2024-05-08T06:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 8 Packages 2024-05-08T06:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 226 M 2024-05-08T06:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T06:34:19.864 INFO:teuthology.orchestra.run.smithi163.stdout:(24/156): ceph-test-19.0.0-3538.gcd693b0b.el9.x 15 MB/s | 49 MB 00:03 2024-05-08T06:34:19.890 INFO:teuthology.orchestra.run.smithi163.stdout:(25/156): python3-rgw-19.0.0-3538.gcd693b0b.el9 192 kB/s | 99 kB 00:00 2024-05-08T06:34:19.932 INFO:teuthology.orchestra.run.smithi163.stdout:(26/156): rbd-fuse-19.0.0-3538.gcd693b0b.el9.x8 1.3 MB/s | 86 kB 00:00 2024-05-08T06:34:20.132 INFO:teuthology.orchestra.run.smithi163.stdout:(27/156): ceph-radosgw-19.0.0-3538.gcd693b0b.el 2.5 MB/s | 11 MB 00:04 2024-05-08T06:34:20.157 INFO:teuthology.orchestra.run.smithi163.stdout:(28/156): rbd-nbd-19.0.0-3538.gcd693b0b.el9.x86 758 kB/s | 171 kB 00:00 2024-05-08T06:34:20.208 INFO:teuthology.orchestra.run.smithi163.stdout:(29/156): rbd-mirror-19.0.0-3538.gcd693b0b.el9. 9.7 MB/s | 3.1 MB 00:00 2024-05-08T06:34:20.225 INFO:teuthology.orchestra.run.smithi163.stdout:(30/156): ceph-grafana-dashboards-19.0.0-3538.g 372 kB/s | 34 kB 00:00 2024-05-08T06:34:20.250 INFO:teuthology.orchestra.run.smithi163.stdout:(31/156): ceph-mgr-cephadm-19.0.0-3538.gcd693b0 1.5 MB/s | 142 kB 00:00 2024-05-08T06:34:20.392 INFO:teuthology.orchestra.run.smithi163.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3538.gcd 1.9 MB/s | 274 kB 00:00 2024-05-08T06:34:20.450 INFO:teuthology.orchestra.run.smithi163.stdout:(33/156): ceph-mgr-dashboard-19.0.0-3538.gcd693 15 MB/s | 3.5 MB 00:00 2024-05-08T06:34:20.467 INFO:teuthology.orchestra.run.smithi163.stdout:(34/156): ceph-mgr-rook-19.0.0-3538.gcd693b0b.e 656 kB/s | 49 kB 00:00 2024-05-08T06:34:20.518 INFO:teuthology.orchestra.run.smithi163.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3538.gc 246 kB/s | 16 kB 00:00 2024-05-08T06:34:20.543 INFO:teuthology.orchestra.run.smithi163.stdout:(36/156): ceph-volume-19.0.0-3538.gcd693b0b.el9 3.6 MB/s | 273 kB 00:00 2024-05-08T06:34:20.660 INFO:teuthology.orchestra.run.smithi163.stdout:(37/156): cephadm-19.0.0-3538.gcd693b0b.el9.noa 5.3 MB/s | 764 kB 00:00 2024-05-08T06:34:20.677 INFO:teuthology.orchestra.run.smithi163.stdout:(38/156): cephfs-top-19.0.0-3538.gcd693b0b.el9. 136 kB/s | 18 kB 00:00 2024-05-08T06:34:20.692 INFO:teuthology.orchestra.run.smithi192.stdout:(1/156): ceph-19.0.0-3538.gcd693b0b.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-05-08T06:34:21.050 INFO:teuthology.orchestra.run.smithi192.stdout:(2/156): ceph-fuse-19.0.0-3538.gcd693b0b.el9.x8 2.4 MB/s | 891 kB 00:00 2024-05-08T06:34:21.102 INFO:teuthology.orchestra.run.smithi163.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 182 kB/s | 80 kB 00:00 2024-05-08T06:34:21.150 INFO:teuthology.orchestra.run.smithi192.stdout:(3/156): ceph-immutable-object-cache-19.0.0-353 1.4 MB/s | 145 kB 00:00 2024-05-08T06:34:21.594 INFO:teuthology.orchestra.run.smithi163.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 397 kB/s | 195 kB 00:00 2024-05-08T06:34:21.867 INFO:teuthology.orchestra.run.smithi192.stdout:(4/156): ceph-base-19.0.0-3538.gcd693b0b.el9.x8 3.8 MB/s | 5.4 MB 00:01 2024-05-08T06:34:21.869 INFO:teuthology.orchestra.run.smithi163.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 674 kB/s | 803 kB 00:01 2024-05-08T06:34:21.909 INFO:teuthology.orchestra.run.smithi192.stdout:(5/156): ceph-mds-19.0.0-3538.gcd693b0b.el9.x86 3.1 MB/s | 2.4 MB 00:00 2024-05-08T06:34:22.135 INFO:teuthology.orchestra.run.smithi192.stdout:(6/156): ceph-common-19.0.0-3538.gcd693b0b.el9. 13 MB/s | 21 MB 00:01 2024-05-08T06:34:22.170 INFO:teuthology.orchestra.run.smithi163.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 440 kB/s | 253 kB 00:00 2024-05-08T06:34:22.210 INFO:teuthology.orchestra.run.smithi192.stdout:(7/156): ceph-mgr-19.0.0-3538.gcd693b0b.el9.x86 4.4 MB/s | 1.5 MB 00:00 2024-05-08T06:34:22.594 INFO:teuthology.orchestra.run.smithi163.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 251 kB/s | 106 kB 00:00 2024-05-08T06:34:22.795 INFO:teuthology.orchestra.run.smithi163.stdout:(44/156): ceph-mgr-diskprediction-local-19.0.0- 2.9 MB/s | 7.4 MB 00:02 2024-05-08T06:34:22.995 INFO:teuthology.orchestra.run.smithi163.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2024-05-08T06:34:23.020 INFO:teuthology.orchestra.run.smithi163.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 318 kB/s | 135 kB 00:00 2024-05-08T06:34:23.093 INFO:teuthology.orchestra.run.smithi192.stdout:(8/156): ceph-mon-19.0.0-3538.gcd693b0b.el9.x86 4.0 MB/s | 4.7 MB 00:01 2024-05-08T06:34:23.129 INFO:teuthology.orchestra.run.smithi163.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 6.2 MB/s | 689 kB 00:00 2024-05-08T06:34:23.230 INFO:teuthology.orchestra.run.smithi163.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 9.2 MB/s | 943 kB 00:00 2024-05-08T06:34:23.269 INFO:teuthology.orchestra.run.smithi192.stdout:(9/156): ceph-osd-19.0.0-3538.gcd693b0b.el9.x86 15 MB/s | 17 MB 00:01 2024-05-08T06:34:23.294 INFO:teuthology.orchestra.run.smithi192.stdout:(10/156): ceph-selinux-19.0.0-3538.gcd693b0b.el 125 kB/s | 25 kB 00:00 2024-05-08T06:34:23.296 INFO:teuthology.orchestra.run.smithi163.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-08T06:34:23.330 INFO:teuthology.orchestra.run.smithi163.stdout:(50/156): python3-requests-2.25.1-8.el9.noarch. 233 kB/s | 125 kB 00:00 2024-05-08T06:34:23.372 INFO:teuthology.orchestra.run.smithi163.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 286 kB/s | 21 kB 00:00 2024-05-08T06:34:23.386 INFO:teuthology.orchestra.run.smithi192.stdout:(11/156): cephfs-mirror-19.0.0-3538.gcd693b0b.e 2.4 MB/s | 225 kB 00:00 2024-05-08T06:34:23.422 INFO:teuthology.orchestra.run.smithi163.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-05-08T06:34:23.448 INFO:teuthology.orchestra.run.smithi163.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 435 kB/s | 33 kB 00:00 2024-05-08T06:34:23.486 INFO:teuthology.orchestra.run.smithi192.stdout:(12/156): cephfs-shell-19.0.0-3538.gcd693b0b.el 269 kB/s | 27 kB 00:00 2024-05-08T06:34:23.498 INFO:teuthology.orchestra.run.smithi163.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-08T06:34:23.540 INFO:teuthology.orchestra.run.smithi163.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-05-08T06:34:23.574 INFO:teuthology.orchestra.run.smithi163.stdout:(56/156): python3-urllib3-1.26.5-5.el9.noarch.r 371 kB/s | 215 kB 00:00 2024-05-08T06:34:23.578 INFO:teuthology.orchestra.run.smithi192.stdout:(13/156): libcephfs-devel-19.0.0-3538.gcd693b0b 346 kB/s | 32 kB 00:00 2024-05-08T06:34:23.624 INFO:teuthology.orchestra.run.smithi163.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-05-08T06:34:23.666 INFO:teuthology.orchestra.run.smithi163.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-08T06:34:23.691 INFO:teuthology.orchestra.run.smithi163.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 386 kB/s | 45 kB 00:00 2024-05-08T06:34:23.741 INFO:teuthology.orchestra.run.smithi163.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 511 kB/s | 25 kB 00:00 2024-05-08T06:34:23.767 INFO:teuthology.orchestra.run.smithi163.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-05-08T06:34:23.809 INFO:teuthology.orchestra.run.smithi163.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-05-08T06:34:23.826 INFO:teuthology.orchestra.run.smithi163.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-05-08T06:34:23.828 INFO:teuthology.orchestra.run.smithi192.stdout:(14/156): libcephfs2-19.0.0-3538.gcd693b0b.el9. 2.9 MB/s | 746 kB 00:00 2024-05-08T06:34:23.876 INFO:teuthology.orchestra.run.smithi163.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 707 kB/s | 35 kB 00:00 2024-05-08T06:34:23.901 INFO:teuthology.orchestra.run.smithi163.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-08T06:34:23.929 INFO:teuthology.orchestra.run.smithi192.stdout:(15/156): libcephsqlite-19.0.0-3538.gcd693b0b.e 1.6 MB/s | 163 kB 00:00 2024-05-08T06:34:23.935 INFO:teuthology.orchestra.run.smithi163.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-05-08T06:34:24.038 INFO:teuthology.orchestra.run.smithi192.stdout:(16/156): ceph-radosgw-19.0.0-3538.gcd693b0b.el 5.8 MB/s | 11 MB 00:01 2024-05-08T06:34:24.054 INFO:teuthology.orchestra.run.smithi192.stdout:(17/156): librados-devel-19.0.0-3538.gcd693b0b. 1.0 MB/s | 127 kB 00:00 2024-05-08T06:34:24.077 INFO:teuthology.orchestra.run.smithi163.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-08T06:34:24.169 INFO:teuthology.orchestra.run.smithi163.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-05-08T06:34:24.171 INFO:teuthology.orchestra.run.smithi192.stdout:(18/156): libradosstriper1-19.0.0-3538.gcd693b0 3.7 MB/s | 500 kB 00:00 2024-05-08T06:34:24.227 INFO:teuthology.orchestra.run.smithi163.stdout:(69/156): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-05-08T06:34:24.272 INFO:teuthology.orchestra.run.smithi192.stdout:(19/156): python3-ceph-argparse-19.0.0-3538.gcd 452 kB/s | 45 kB 00:00 2024-05-08T06:34:24.277 INFO:teuthology.orchestra.run.smithi163.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 4.4 MB/s | 911 kB 00:00 2024-05-08T06:34:24.311 INFO:teuthology.orchestra.run.smithi163.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 937 kB/s | 133 kB 00:00 2024-05-08T06:34:24.370 INFO:teuthology.orchestra.run.smithi163.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-08T06:34:24.372 INFO:teuthology.orchestra.run.smithi192.stdout:(20/156): python3-ceph-common-19.0.0-3538.gcd69 1.3 MB/s | 135 kB 00:00 2024-05-08T06:34:24.420 INFO:teuthology.orchestra.run.smithi163.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-05-08T06:34:24.472 INFO:teuthology.orchestra.run.smithi192.stdout:(21/156): python3-cephfs-19.0.0-3538.gcd693b0b. 1.6 MB/s | 161 kB 00:00 2024-05-08T06:34:24.545 INFO:teuthology.orchestra.run.smithi163.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-05-08T06:34:24.579 INFO:teuthology.orchestra.run.smithi163.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-05-08T06:34:24.581 INFO:teuthology.orchestra.run.smithi192.stdout:(22/156): python3-rados-19.0.0-3538.gcd693b0b.e 2.9 MB/s | 320 kB 00:00 2024-05-08T06:34:24.638 INFO:teuthology.orchestra.run.smithi163.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-05-08T06:34:24.680 INFO:teuthology.orchestra.run.smithi163.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-05-08T06:34:24.698 INFO:teuthology.orchestra.run.smithi192.stdout:(23/156): python3-rbd-19.0.0-3538.gcd693b0b.el9 2.5 MB/s | 301 kB 00:00 2024-05-08T06:34:24.733 INFO:teuthology.orchestra.run.smithi163.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 202 kB/s | 19 kB 00:00 2024-05-08T06:34:24.798 INFO:teuthology.orchestra.run.smithi192.stdout:(24/156): python3-rgw-19.0.0-3538.gcd693b0b.el9 996 kB/s | 99 kB 00:00 2024-05-08T06:34:24.806 INFO:teuthology.orchestra.run.smithi163.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-05-08T06:34:24.831 INFO:teuthology.orchestra.run.smithi163.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-05-08T06:34:24.856 INFO:teuthology.orchestra.run.smithi163.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 284 kB/s | 35 kB 00:00 2024-05-08T06:34:24.898 INFO:teuthology.orchestra.run.smithi192.stdout:(25/156): rbd-fuse-19.0.0-3538.gcd693b0b.el9.x8 860 kB/s | 86 kB 00:00 2024-05-08T06:34:24.923 INFO:teuthology.orchestra.run.smithi163.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-05-08T06:34:24.957 INFO:teuthology.orchestra.run.smithi163.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-08T06:34:25.015 INFO:teuthology.orchestra.run.smithi163.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 837 kB/s | 77 kB 00:00 2024-05-08T06:34:25.040 INFO:teuthology.orchestra.run.smithi163.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 506 kB/s | 42 kB 00:00 2024-05-08T06:34:25.141 INFO:teuthology.orchestra.run.smithi163.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-05-08T06:34:25.174 INFO:teuthology.orchestra.run.smithi192.stdout:(26/156): librgw2-19.0.0-3538.gcd693b0b.el9.x86 4.8 MB/s | 5.4 MB 00:01 2024-05-08T06:34:25.183 INFO:teuthology.orchestra.run.smithi163.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-05-08T06:34:25.208 INFO:teuthology.orchestra.run.smithi163.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 1.4 MB/s | 269 kB 00:00 2024-05-08T06:34:25.258 INFO:teuthology.orchestra.run.smithi163.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-05-08T06:34:25.284 INFO:teuthology.orchestra.run.smithi163.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 506 kB/s | 51 kB 00:00 2024-05-08T06:34:25.291 INFO:teuthology.orchestra.run.smithi192.stdout:(27/156): rbd-nbd-19.0.0-3538.gcd693b0b.el9.x86 1.4 MB/s | 171 kB 00:00 2024-05-08T06:34:25.309 INFO:teuthology.orchestra.run.smithi163.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 533 kB/s | 54 kB 00:00 2024-05-08T06:34:25.359 INFO:teuthology.orchestra.run.smithi163.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-05-08T06:34:25.384 INFO:teuthology.orchestra.run.smithi163.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 587 kB/s | 44 kB 00:00 2024-05-08T06:34:25.391 INFO:teuthology.orchestra.run.smithi192.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3538.g 343 kB/s | 34 kB 00:00 2024-05-08T06:34:25.451 INFO:teuthology.orchestra.run.smithi163.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-05-08T06:34:25.483 INFO:teuthology.orchestra.run.smithi192.stdout:(29/156): rbd-mirror-19.0.0-3538.gcd693b0b.el9. 5.3 MB/s | 3.1 MB 00:00 2024-05-08T06:34:25.485 INFO:teuthology.orchestra.run.smithi163.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 359 kB/s | 36 kB 00:00 2024-05-08T06:34:25.508 INFO:teuthology.orchestra.run.smithi192.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3538.gcd693b0 1.2 MB/s | 142 kB 00:00 2024-05-08T06:34:25.535 INFO:teuthology.orchestra.run.smithi163.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-05-08T06:34:25.586 INFO:teuthology.orchestra.run.smithi163.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-08T06:34:25.611 INFO:teuthology.orchestra.run.smithi163.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 768 kB/s | 58 kB 00:00 2024-05-08T06:34:25.670 INFO:teuthology.orchestra.run.smithi163.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-08T06:34:25.703 INFO:teuthology.orchestra.run.smithi163.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 799 kB/s | 94 kB 00:00 2024-05-08T06:34:25.753 INFO:teuthology.orchestra.run.smithi163.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 536 kB/s | 45 kB 00:00 2024-05-08T06:34:25.975 INFO:teuthology.orchestra.run.smithi192.stdout:(31/156): ceph-test-19.0.0-3538.gcd693b0b.el9.x 18 MB/s | 49 MB 00:02 2024-05-08T06:34:25.996 INFO:teuthology.orchestra.run.smithi163.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-05-08T06:34:26.012 INFO:teuthology.orchestra.run.smithi163.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 72 kB/s | 22 kB 00:00 2024-05-08T06:34:26.051 INFO:teuthology.orchestra.run.smithi192.stdout:(32/156): ceph-mgr-dashboard-19.0.0-3538.gcd693 6.2 MB/s | 3.5 MB 00:00 2024-05-08T06:34:26.076 INFO:teuthology.orchestra.run.smithi192.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3538.gcd 2.7 MB/s | 274 kB 00:00 2024-05-08T06:34:26.101 INFO:teuthology.orchestra.run.smithi192.stdout:(34/156): ceph-mgr-rook-19.0.0-3538.gcd693b0b.e 988 kB/s | 49 kB 00:00 2024-05-08T06:34:26.118 INFO:teuthology.orchestra.run.smithi192.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3538.gc 394 kB/s | 16 kB 00:00 2024-05-08T06:34:26.168 INFO:teuthology.orchestra.run.smithi192.stdout:(36/156): ceph-volume-19.0.0-3538.gcd693b0b.el9 4.0 MB/s | 273 kB 00:00 2024-05-08T06:34:26.202 INFO:teuthology.orchestra.run.smithi192.stdout:(37/156): cephadm-19.0.0-3538.gcd693b0b.el9.noa 8.9 MB/s | 764 kB 00:00 2024-05-08T06:34:26.227 INFO:teuthology.orchestra.run.smithi192.stdout:(38/156): cephfs-top-19.0.0-3538.gcd693b0b.el9. 312 kB/s | 18 kB 00:00 2024-05-08T06:34:26.230 INFO:teuthology.orchestra.run.smithi163.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-05-08T06:34:26.472 INFO:teuthology.orchestra.run.smithi163.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-05-08T06:34:26.555 INFO:teuthology.orchestra.run.smithi163.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 569 kB/s | 309 kB 00:00 2024-05-08T06:34:26.614 INFO:teuthology.orchestra.run.smithi163.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 887 kB/s | 548 kB 00:00 2024-05-08T06:34:26.647 INFO:teuthology.orchestra.run.smithi163.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-05-08T06:34:26.706 INFO:teuthology.orchestra.run.smithi163.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-05-08T06:34:26.756 INFO:teuthology.orchestra.run.smithi163.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-05-08T06:34:26.811 INFO:teuthology.orchestra.run.smithi192.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 5.7 MB/s | 7.4 MB 00:01 2024-05-08T06:34:26.815 INFO:teuthology.orchestra.run.smithi163.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-05-08T06:34:26.965 INFO:teuthology.orchestra.run.smithi163.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-05-08T06:34:26.999 INFO:teuthology.orchestra.run.smithi163.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-05-08T06:34:27.066 INFO:teuthology.orchestra.run.smithi163.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 432 kB/s | 43 kB 00:00 2024-05-08T06:34:27.091 INFO:teuthology.orchestra.run.smithi163.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-05-08T06:34:27.183 INFO:teuthology.orchestra.run.smithi163.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-05-08T06:34:27.208 INFO:teuthology.orchestra.run.smithi163.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-05-08T06:34:27.253 INFO:teuthology.orchestra.run.smithi192.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 76 kB/s | 80 kB 00:01 2024-05-08T06:34:27.267 INFO:teuthology.orchestra.run.smithi163.stdout:(118/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-05-08T06:34:27.325 INFO:teuthology.orchestra.run.smithi163.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-05-08T06:34:27.351 INFO:teuthology.orchestra.run.smithi163.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-05-08T06:34:27.393 INFO:teuthology.orchestra.run.smithi163.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 281 kB/s | 35 kB 00:00 2024-05-08T06:34:27.526 INFO:teuthology.orchestra.run.smithi163.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-05-08T06:34:27.652 INFO:teuthology.orchestra.run.smithi163.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 86 kB/s | 11 kB 00:00 2024-05-08T06:34:27.677 INFO:teuthology.orchestra.run.smithi163.stdout:(124/156): python3-google-auth-2.29.0-1.el9.noa 585 kB/s | 206 kB 00:00 2024-05-08T06:34:27.727 INFO:teuthology.orchestra.run.smithi163.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-05-08T06:34:27.752 INFO:teuthology.orchestra.run.smithi163.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-08T06:34:27.778 INFO:teuthology.orchestra.run.smithi163.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-05-08T06:34:27.828 INFO:teuthology.orchestra.run.smithi163.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-05-08T06:34:27.828 INFO:teuthology.orchestra.run.smithi192.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 502 kB/s | 803 kB 00:01 2024-05-08T06:34:27.853 INFO:teuthology.orchestra.run.smithi163.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-05-08T06:34:27.912 INFO:teuthology.orchestra.run.smithi163.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-05-08T06:34:27.946 INFO:teuthology.orchestra.run.smithi163.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 395 kB/s | 46 kB 00:00 2024-05-08T06:34:27.980 INFO:teuthology.orchestra.run.smithi163.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 626 kB/s | 79 kB 00:00 2024-05-08T06:34:28.046 INFO:teuthology.orchestra.run.smithi163.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 430 kB/s | 58 kB 00:00 2024-05-08T06:34:28.088 INFO:teuthology.orchestra.run.smithi163.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-05-08T06:34:28.113 INFO:teuthology.orchestra.run.smithi163.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 123 kB/s | 16 kB 00:00 2024-05-08T06:34:28.172 INFO:teuthology.orchestra.run.smithi163.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 717 kB/s | 90 kB 00:00 2024-05-08T06:34:28.198 INFO:teuthology.orchestra.run.smithi163.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-05-08T06:34:28.212 INFO:teuthology.orchestra.run.smithi192.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 139 kB/s | 195 kB 00:01 2024-05-08T06:34:28.223 INFO:teuthology.orchestra.run.smithi163.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2024-05-08T06:34:28.281 INFO:teuthology.orchestra.run.smithi163.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-08T06:34:28.315 INFO:teuthology.orchestra.run.smithi163.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 505 kB/s | 59 kB 00:00 2024-05-08T06:34:28.340 INFO:teuthology.orchestra.run.smithi163.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2024-05-08T06:34:28.374 INFO:teuthology.orchestra.run.smithi163.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 554 kB/s | 51 kB 00:00 2024-05-08T06:34:28.416 INFO:teuthology.orchestra.run.smithi163.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-08T06:34:28.441 INFO:teuthology.orchestra.run.smithi163.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-05-08T06:34:28.475 INFO:teuthology.orchestra.run.smithi163.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-05-08T06:34:28.517 INFO:teuthology.orchestra.run.smithi163.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 198 kB/s | 20 kB 00:00 2024-05-08T06:34:28.545 INFO:teuthology.orchestra.run.smithi192.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 196 kB/s | 253 kB 00:01 2024-05-08T06:34:28.567 INFO:teuthology.orchestra.run.smithi163.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-05-08T06:34:28.609 INFO:teuthology.orchestra.run.smithi163.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-08T06:34:28.967 INFO:teuthology.orchestra.run.smithi163.stdout:(149/156): librbd1-19.0.0-3538.gcd693b0b.el9.x8 7.9 MB/s | 3.1 MB 00:00 2024-05-08T06:34:29.089 INFO:teuthology.orchestra.run.smithi192.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 126 kB/s | 106 kB 00:00 2024-05-08T06:34:29.146 INFO:teuthology.orchestra.run.smithi192.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 954 kB/s | 1.2 MB 00:01 2024-05-08T06:34:29.351 INFO:teuthology.orchestra.run.smithi163.stdout:(150/156): librados2-19.0.0-3538.gcd693b0b.el9. 4.1 MB/s | 3.4 MB 00:00 2024-05-08T06:34:29.496 INFO:teuthology.orchestra.run.smithi192.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 142 kB/s | 135 kB 00:00 2024-05-08T06:34:29.693 INFO:teuthology.orchestra.run.smithi163.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 122 kB/s | 42 kB 00:00 2024-05-08T06:34:29.788 INFO:teuthology.orchestra.run.smithi192.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-05-08T06:34:30.010 INFO:teuthology.orchestra.run.smithi163.stdout:(152/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.8 MB/s | 1.8 MB 00:01 2024-05-08T06:34:30.055 INFO:teuthology.orchestra.run.smithi192.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 943 kB 00:00 2024-05-08T06:34:30.105 INFO:teuthology.orchestra.run.smithi192.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-05-08T06:34:30.130 INFO:teuthology.orchestra.run.smithi192.stdout:(50/156): python3-requests-2.25.1-8.el9.noarch. 120 kB/s | 125 kB 00:01 2024-05-08T06:34:30.156 INFO:teuthology.orchestra.run.smithi192.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 439 kB/s | 21 kB 00:00 2024-05-08T06:34:30.206 INFO:teuthology.orchestra.run.smithi192.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-05-08T06:34:30.256 INFO:teuthology.orchestra.run.smithi192.stdout:(53/156): python3-urllib3-1.26.5-5.el9.noarch.r 193 kB/s | 215 kB 00:01 2024-05-08T06:34:30.323 INFO:teuthology.orchestra.run.smithi192.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 1.6 MB/s | 313 kB 00:00 2024-05-08T06:34:30.349 INFO:teuthology.orchestra.run.smithi192.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 199 kB/s | 18 kB 00:00 2024-05-08T06:34:30.374 INFO:teuthology.orchestra.run.smithi192.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-05-08T06:34:30.424 INFO:teuthology.orchestra.run.smithi192.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-05-08T06:34:30.427 INFO:teuthology.orchestra.run.smithi163.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 339 kB/s | 248 kB 00:00 2024-05-08T06:34:30.458 INFO:teuthology.orchestra.run.smithi192.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-08T06:34:30.566 INFO:teuthology.orchestra.run.smithi192.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-05-08T06:34:30.608 INFO:teuthology.orchestra.run.smithi192.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-05-08T06:34:30.650 INFO:teuthology.orchestra.run.smithi192.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-05-08T06:34:30.667 INFO:teuthology.orchestra.run.smithi192.stdout:(62/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 253 kB/s | 25 kB 00:00 2024-05-08T06:34:30.759 INFO:teuthology.orchestra.run.smithi192.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-08T06:34:30.784 INFO:teuthology.orchestra.run.smithi192.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-08T06:34:30.801 INFO:teuthology.orchestra.run.smithi192.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 839 kB/s | 35 kB 00:00 2024-05-08T06:34:31.010 INFO:teuthology.orchestra.run.smithi192.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.7 MB/s | 3.0 MB 00:00 2024-05-08T06:34:31.335 INFO:teuthology.orchestra.run.smithi192.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-05-08T06:34:31.502 INFO:teuthology.orchestra.run.smithi163.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-05-08T06:34:31.535 INFO:teuthology.orchestra.run.smithi192.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 4.5 MB/s | 911 kB 00:00 2024-05-08T06:34:31.611 INFO:teuthology.orchestra.run.smithi192.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-05-08T06:34:31.628 INFO:teuthology.orchestra.run.smithi192.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-08T06:34:31.636 INFO:teuthology.orchestra.run.smithi163.stdout:(155/156): NetworkManager-1.47.5-1.el9.x86_64.r 781 kB/s | 2.3 MB 00:03 2024-05-08T06:34:31.831 INFO:teuthology.orchestra.run.smithi192.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-05-08T06:34:31.895 INFO:teuthology.orchestra.run.smithi192.stdout:(72/156): openblas-openmp-0.3.21-2.el9.x86_64.r 4.3 MB/s | 4.7 MB 00:01 2024-05-08T06:34:31.920 INFO:teuthology.orchestra.run.smithi192.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 951 kB/s | 87 kB 00:00 2024-05-08T06:34:32.004 INFO:teuthology.orchestra.run.smithi192.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-08T06:34:32.079 INFO:teuthology.orchestra.run.smithi192.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-05-08T06:34:32.130 INFO:teuthology.orchestra.run.smithi192.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-05-08T06:34:32.180 INFO:teuthology.orchestra.run.smithi192.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-05-08T06:34:32.239 INFO:teuthology.orchestra.run.smithi192.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-08T06:34:32.297 INFO:teuthology.orchestra.run.smithi192.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 595 kB/s | 35 kB 00:00 2024-05-08T06:34:32.352 INFO:teuthology.orchestra.run.smithi163.stdout:(156/156): openssl-3.2.1-1.el9.x86_64.rpm 590 kB/s | 1.3 MB 00:02 2024-05-08T06:34:32.356 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-05-08T06:34:32.357 INFO:teuthology.orchestra.run.smithi163.stdout:Total 12 MB/s | 226 MB 00:18 2024-05-08T06:34:32.831 INFO:teuthology.orchestra.run.smithi192.stdout:(80/156): postgresql-server-13.14-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-05-08T06:34:32.915 INFO:teuthology.orchestra.run.smithi192.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-05-08T06:34:33.006 INFO:teuthology.orchestra.run.smithi192.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-05-08T06:34:33.057 INFO:teuthology.orchestra.run.smithi192.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-08T06:34:33.140 INFO:teuthology.orchestra.run.smithi192.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.0 MB/s | 5.1 MB 00:00 2024-05-08T06:34:33.157 INFO:teuthology.orchestra.run.smithi192.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-05-08T06:34:33.216 INFO:teuthology.orchestra.run.smithi192.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-08T06:34:33.241 INFO:teuthology.orchestra.run.smithi192.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-05-08T06:34:33.383 INFO:teuthology.orchestra.run.smithi192.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 359 kB/s | 51 kB 00:00 2024-05-08T06:34:33.408 INFO:teuthology.orchestra.run.smithi192.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.4 MB/s | 279 kB 00:00 2024-05-08T06:34:33.434 INFO:teuthology.orchestra.run.smithi192.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-08T06:34:33.484 INFO:teuthology.orchestra.run.smithi192.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-05-08T06:34:33.540 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-05-08T06:34:33.542 INFO:teuthology.orchestra.run.smithi192.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-05-08T06:34:33.601 INFO:teuthology.orchestra.run.smithi192.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-08T06:34:33.660 INFO:teuthology.orchestra.run.smithi192.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-05-08T06:34:33.689 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-05-08T06:34:33.689 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-05-08T06:34:33.719 INFO:teuthology.orchestra.run.smithi192.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-05-08T06:34:33.803 INFO:teuthology.orchestra.run.smithi192.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-05-08T06:34:33.870 INFO:teuthology.orchestra.run.smithi192.stdout:(97/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.0 MB/s | 6.0 MB 00:01 2024-05-08T06:34:33.895 INFO:teuthology.orchestra.run.smithi192.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 629 kB/s | 58 kB 00:00 2024-05-08T06:34:33.920 INFO:teuthology.orchestra.run.smithi192.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-08T06:34:33.954 INFO:teuthology.orchestra.run.smithi192.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-08T06:34:33.979 INFO:teuthology.orchestra.run.smithi192.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-05-08T06:34:34.096 INFO:teuthology.orchestra.run.smithi192.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 157 kB/s | 22 kB 00:00 2024-05-08T06:34:34.246 INFO:teuthology.orchestra.run.smithi192.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.2 MB/s | 863 kB 00:00 2024-05-08T06:34:34.372 INFO:teuthology.orchestra.run.smithi192.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-05-08T06:34:34.414 INFO:teuthology.orchestra.run.smithi192.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 466 kB/s | 19 kB 00:00 2024-05-08T06:34:34.489 INFO:teuthology.orchestra.run.smithi192.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-05-08T06:34:34.531 INFO:teuthology.orchestra.run.smithi192.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 625 kB/s | 26 kB 00:00 2024-05-08T06:34:34.573 INFO:teuthology.orchestra.run.smithi192.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-05-08T06:34:34.615 INFO:teuthology.orchestra.run.smithi192.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-05-08T06:34:34.673 INFO:teuthology.orchestra.run.smithi192.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-08T06:34:34.841 INFO:teuthology.orchestra.run.smithi192.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-05-08T06:34:34.949 INFO:teuthology.orchestra.run.smithi192.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-05-08T06:34:34.991 INFO:teuthology.orchestra.run.smithi192.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-05-08T06:34:35.033 INFO:teuthology.orchestra.run.smithi192.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 773 kB/s | 32 kB 00:00 2024-05-08T06:34:35.075 INFO:teuthology.orchestra.run.smithi192.stdout:(115/156): python3-certifi-2021.10.8-2.el9.noar 359 kB/s | 15 kB 00:00 2024-05-08T06:34:35.125 INFO:teuthology.orchestra.run.smithi192.stdout:(116/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-05-08T06:34:35.209 INFO:teuthology.orchestra.run.smithi192.stdout:(117/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-08T06:34:35.284 INFO:teuthology.orchestra.run.smithi192.stdout:(118/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-05-08T06:34:35.336 INFO:teuthology.orchestra.run.smithi192.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.3 MB/s | 303 kB 00:00 2024-05-08T06:34:35.360 INFO:teuthology.orchestra.run.smithi192.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 468 kB/s | 35 kB 00:00 2024-05-08T06:34:35.402 INFO:teuthology.orchestra.run.smithi192.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 3.1 MB/s | 206 kB 00:00 2024-05-08T06:34:35.460 INFO:teuthology.orchestra.run.smithi192.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-08T06:34:35.502 INFO:teuthology.orchestra.run.smithi192.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 255 kB/s | 11 kB 00:00 2024-05-08T06:34:35.545 INFO:teuthology.orchestra.run.smithi192.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 424 kB/s | 18 kB 00:00 2024-05-08T06:34:35.586 INFO:teuthology.orchestra.run.smithi192.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 557 kB/s | 23 kB 00:00 2024-05-08T06:34:35.629 INFO:teuthology.orchestra.run.smithi192.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 467 kB/s | 19 kB 00:00 2024-05-08T06:34:35.670 INFO:teuthology.orchestra.run.smithi192.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 473 kB/s | 20 kB 00:00 2024-05-08T06:34:35.871 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-05-08T06:34:35.872 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-05-08T06:34:35.979 INFO:teuthology.orchestra.run.smithi192.stdout:(128/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.3 MB/s | 16 MB 00:02 2024-05-08T06:34:36.013 INFO:teuthology.orchestra.run.smithi192.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-05-08T06:34:36.080 INFO:teuthology.orchestra.run.smithi192.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-05-08T06:34:36.122 INFO:teuthology.orchestra.run.smithi192.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-05-08T06:34:36.172 INFO:teuthology.orchestra.run.smithi192.stdout:(132/156): python3-logutils-0.3.5-21.el9.noarch 241 kB/s | 46 kB 00:00 2024-05-08T06:34:36.214 INFO:teuthology.orchestra.run.smithi192.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-05-08T06:34:36.239 INFO:teuthology.orchestra.run.smithi192.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-05-08T06:34:36.281 INFO:teuthology.orchestra.run.smithi192.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-08T06:34:36.323 INFO:teuthology.orchestra.run.smithi192.stdout:(136/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.1 MB/s | 2.0 MB 00:00 2024-05-08T06:34:36.348 INFO:teuthology.orchestra.run.smithi192.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-05-08T06:34:36.373 INFO:teuthology.orchestra.run.smithi192.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-05-08T06:34:36.407 INFO:teuthology.orchestra.run.smithi192.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-08T06:34:36.432 INFO:teuthology.orchestra.run.smithi192.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-05-08T06:34:36.457 INFO:teuthology.orchestra.run.smithi192.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-08T06:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-05-08T06:34:36.525 INFO:teuthology.orchestra.run.smithi192.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-05-08T06:34:36.567 INFO:teuthology.orchestra.run.smithi192.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 480 kB/s | 20 kB 00:00 2024-05-08T06:34:36.592 INFO:teuthology.orchestra.run.smithi192.stdout:(145/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-05-08T06:34:36.617 INFO:teuthology.orchestra.run.smithi192.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-05-08T06:34:36.659 INFO:teuthology.orchestra.run.smithi192.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-08T06:34:37.110 INFO:teuthology.orchestra.run.smithi192.stdout:(148/156): librbd1-19.0.0-3538.gcd693b0b.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-05-08T06:34:37.160 INFO:teuthology.orchestra.run.smithi192.stdout:(149/156): librados2-19.0.0-3538.gcd693b0b.el9. 6.3 MB/s | 3.4 MB 00:00 2024-05-08T06:34:37.352 INFO:teuthology.orchestra.run.smithi192.stdout:(150/156): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-05-08T06:34:38.235 INFO:teuthology.orchestra.run.smithi192.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 47 kB/s | 42 kB 00:00 2024-05-08T06:34:38.677 INFO:teuthology.orchestra.run.smithi192.stdout:(152/156): NetworkManager-1.47.5-1.el9.x86_64.r 1.5 MB/s | 2.3 MB 00:01 2024-05-08T06:34:38.720 INFO:teuthology.orchestra.run.smithi192.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.2 MB/s | 1.8 MB 00:01 2024-05-08T06:34:39.470 INFO:teuthology.orchestra.run.smithi192.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 201 kB/s | 248 kB 00:01 2024-05-08T06:34:40.103 INFO:teuthology.orchestra.run.smithi192.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 970 kB/s | 1.3 MB 00:01 2024-05-08T06:34:40.203 INFO:teuthology.orchestra.run.smithi192.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.6 MB/s | 2.4 MB 00:01 2024-05-08T06:34:40.212 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T06:34:40.212 INFO:teuthology.orchestra.run.smithi192.stdout:Total 11 MB/s | 226 MB 00:20 2024-05-08T06:34:41.417 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T06:34:41.570 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T06:34:41.570 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T06:34:41.952 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-05-08T06:34:42.033 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T06:34:42.070 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T06:34:42.134 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T06:34:42.174 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T06:34:42.200 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T06:34:42.252 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T06:34:42.637 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T06:34:42.692 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 8/164 2024-05-08T06:34:42.814 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 8/164 2024-05-08T06:34:42.852 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 9/164 2024-05-08T06:34:42.958 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 9/164 2024-05-08T06:34:42.990 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rados-2:19.0.0-3538.gcd693b0b.el9.x86_64 10/164 2024-05-08T06:34:43.179 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T06:34:43.249 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T06:34:43.273 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 13/164 2024-05-08T06:34:43.377 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 13/164 2024-05-08T06:34:43.399 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86 14/164 2024-05-08T06:34:43.462 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86 14/164 2024-05-08T06:34:43.492 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T06:34:43.510 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T06:34:43.701 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T06:34:43.834 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T06:34:43.834 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T06:34:43.875 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T06:34:44.047 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T06:34:44.091 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T06:34:44.208 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T06:34:44.276 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T06:34:44.484 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T06:34:44.610 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T06:34:44.672 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T06:34:44.714 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T06:34:44.795 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T06:34:44.831 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T06:34:44.883 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T06:34:44.924 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T06:34:44.951 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T06:34:45.019 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T06:34:45.073 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T06:34:45.105 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T06:34:45.148 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T06:34:45.223 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T06:34:45.258 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gcd693b0b.el9. 32/164 2024-05-08T06:34:45.310 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gcd693b0b.el 33/164 2024-05-08T06:34:45.339 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cephfs-2:19.0.0-3538.gcd693b0b.el9.x86_6 34/164 2024-05-08T06:34:45.408 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T06:34:45.462 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T06:34:45.535 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T06:34:45.667 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T06:34:45.722 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T06:34:46.236 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T06:34:46.318 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T06:34:46.348 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T06:34:46.382 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T06:34:46.420 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librados-devel-2:19.0.0-3538.gcd693b0b.el9.x86_6 44/164 2024-05-08T06:34:46.470 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T06:34:46.621 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T06:34:46.690 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T06:34:46.780 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T06:34:46.845 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T06:34:46.925 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T06:34:46.966 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T06:34:47.003 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T06:34:47.049 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T06:34:47.085 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T06:34:47.128 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T06:34:47.170 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T06:34:47.313 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T06:34:47.410 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T06:34:47.440 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T06:34:47.481 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T06:34:47.574 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T06:34:47.610 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T06:34:47.649 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T06:34:47.689 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T06:34:47.714 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T06:34:47.821 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T06:34:47.949 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T06:34:48.004 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T06:34:48.032 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T06:34:48.074 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T06:34:48.114 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T06:34:48.145 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T06:34:48.175 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T06:34:48.218 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T06:34:48.260 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T06:34:48.310 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T06:34:48.344 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T06:34:48.432 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T06:34:48.475 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T06:34:48.514 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T06:34:48.570 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T06:34:48.668 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T06:34:48.724 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T06:34:49.440 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T06:34:49.488 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T06:34:49.530 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T06:34:49.628 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T06:34:49.687 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T06:34:49.777 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T06:34:49.807 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T06:34:49.978 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T06:34:49.993 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T06:34:50.030 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T06:34:50.057 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T06:34:50.096 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T06:34:50.118 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T06:34:50.172 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T06:34:50.208 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T06:34:50.234 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T06:34:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T06:34:50.615 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T06:34:50.735 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 8/164 2024-05-08T06:34:50.766 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T06:34:50.857 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 8/164 2024-05-08T06:34:50.878 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T06:34:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 9/164 2024-05-08T06:34:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 9/164 2024-05-08T06:34:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:19.0.0-3538.gcd693b0b.el9.x86_64 10/164 2024-05-08T06:34:51.212 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T06:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T06:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 13/164 2024-05-08T06:34:51.386 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 13/164 2024-05-08T06:34:51.408 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86 14/164 2024-05-08T06:34:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86 14/164 2024-05-08T06:34:51.484 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T06:34:51.502 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T06:34:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T06:34:51.873 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T06:34:51.998 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T06:34:52.059 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T06:34:52.134 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T06:34:52.181 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T06:34:52.217 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T06:34:52.245 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T06:34:52.268 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T06:34:52.278 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T06:34:52.484 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T06:34:52.598 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T06:34:52.598 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T06:34:52.637 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T06:34:52.649 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 101/164 2024-05-08T06:34:52.673 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T06:34:52.738 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T06:34:52.747 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 101/164 2024-05-08T06:34:52.773 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T06:34:52.779 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 102/164 2024-05-08T06:34:52.815 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T06:34:52.816 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T06:34:52.849 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T06:34:52.877 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T06:34:52.954 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T06:34:53.007 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T06:34:53.038 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T06:34:53.081 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T06:34:53.158 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T06:34:53.193 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gcd693b0b.el9. 32/164 2024-05-08T06:34:53.236 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gcd693b0b.el 33/164 2024-05-08T06:34:53.265 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:19.0.0-3538.gcd693b0b.el9.x86_6 34/164 2024-05-08T06:34:53.296 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T06:34:53.335 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T06:34:53.338 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 105/164 2024-05-08T06:34:53.379 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T06:34:53.407 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 105/164 2024-05-08T06:34:53.437 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 106/164 2024-05-08T06:34:53.454 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T06:34:53.589 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T06:34:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T06:34:54.172 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T06:34:54.253 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T06:34:54.282 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T06:34:54.324 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T06:34:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:19.0.0-3538.gcd693b0b.el9.x86_6 44/164 2024-05-08T06:34:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T06:34:54.555 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T06:34:54.607 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T06:34:54.698 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T06:34:54.754 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T06:34:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T06:34:54.867 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T06:34:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T06:34:54.950 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T06:34:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T06:34:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T06:34:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T06:34:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T06:34:55.252 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T06:34:55.298 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T06:34:55.340 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T06:34:55.435 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T06:34:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T06:34:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T06:34:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T06:34:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T06:34:55.675 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T06:34:55.698 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 107/164 2024-05-08T06:34:55.735 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 107/164 2024-05-08T06:34:55.799 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T06:34:55.847 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T06:34:55.882 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T06:34:55.909 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T06:34:55.948 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T06:34:55.980 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T06:34:56.010 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T06:34:56.044 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T06:34:56.078 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T06:34:56.119 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T06:34:56.154 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T06:34:56.240 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T06:34:56.277 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 107/164 2024-05-08T06:34:56.293 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T06:34:56.314 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 108/164 2024-05-08T06:34:56.324 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T06:34:56.371 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T06:34:56.415 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 108/164 2024-05-08T06:34:56.416 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T06:34:56.416 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T06:34:56.416 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:34:56.457 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 109/164 2024-05-08T06:34:56.469 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T06:34:56.517 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T06:34:57.252 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T06:34:57.298 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T06:34:57.331 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T06:34:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T06:34:57.489 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T06:34:57.578 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T06:34:57.617 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T06:34:57.793 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T06:34:57.822 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T06:34:57.902 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T06:34:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T06:34:58.666 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T06:34:59.929 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T06:35:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T06:35:00.046 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T06:35:00.079 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T06:35:00.399 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T06:35:00.450 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 101/164 2024-05-08T06:35:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 101/164 2024-05-08T06:35:00.555 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 102/164 2024-05-08T06:35:00.592 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T06:35:01.069 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T06:35:01.095 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 105/164 2024-05-08T06:35:01.167 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 105/164 2024-05-08T06:35:01.197 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 106/164 2024-05-08T06:35:03.457 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 107/164 2024-05-08T06:35:03.493 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 107/164 2024-05-08T06:35:04.043 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 107/164 2024-05-08T06:35:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 108/164 2024-05-08T06:35:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 108/164 2024-05-08T06:35:04.165 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T06:35:04.165 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T06:35:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:04.208 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 109/164 2024-05-08T06:35:08.038 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 109/164 2024-05-08T06:35:08.038 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /sys 2024-05-08T06:35:08.038 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /proc 2024-05-08T06:35:08.038 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /mnt 2024-05-08T06:35:08.038 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /var/tmp 2024-05-08T06:35:08.039 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /home 2024-05-08T06:35:08.039 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /root 2024-05-08T06:35:08.039 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /tmp 2024-05-08T06:35:08.039 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:08.209 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 110/164 2024-05-08T06:35:08.225 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 110/164 2024-05-08T06:35:08.225 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:08.225 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T06:35:08.225 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:09.729 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 111/164 2024-05-08T06:35:09.756 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 111/164 2024-05-08T06:35:09.756 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:09.757 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T06:35:09.757 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T06:35:09.757 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T06:35:09.757 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:09.997 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 112/164 2024-05-08T06:35:10.021 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 112/164 2024-05-08T06:35:10.021 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:10.021 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T06:35:10.021 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T06:35:10.021 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T06:35:10.021 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:10.465 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 113/164 2024-05-08T06:35:10.489 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 113/164 2024-05-08T06:35:10.489 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:10.489 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T06:35:10.489 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T06:35:10.489 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T06:35:10.489 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:10.595 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T06:35:10.651 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T06:35:10.707 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T06:35:10.890 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T06:35:10.940 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T06:35:11.009 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T06:35:11.098 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T06:35:11.237 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T06:35:11.288 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T06:35:11.323 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T06:35:11.350 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T06:35:11.812 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 125/164 2024-05-08T06:35:11.840 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 125/164 2024-05-08T06:35:11.872 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gcd693b0b.e 126/164 2024-05-08T06:35:11.934 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gcd693b0b. 127/164 2024-05-08T06:35:11.963 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noa 128/164 2024-05-08T06:35:13.010 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noa 128/164 2024-05-08T06:35:13.040 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.n 129/164 2024-05-08T06:35:14.014 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.n 129/164 2024-05-08T06:35:14.048 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd6 130/164 2024-05-08T06:35:14.160 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd6 130/164 2024-05-08T06:35:14.327 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gcd693b0b.el 131/164 2024-05-08T06:35:14.359 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 132/164 2024-05-08T06:35:14.384 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 132/164 2024-05-08T06:35:14.384 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:14.384 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T06:35:14.384 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T06:35:14.384 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T06:35:14.384 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:14.435 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 133/164 2024-05-08T06:35:14.446 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 133/164 2024-05-08T06:35:14.543 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-2:19.0.0-3538.gcd693b0b.el9.x86_64 134/164 2024-05-08T06:35:14.604 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 135/164 2024-05-08T06:35:15.543 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T06:35:15.570 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 137/164 2024-05-08T06:35:15.594 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 137/164 2024-05-08T06:35:15.594 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:15.594 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T06:35:15.594 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T06:35:15.594 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T06:35:15.594 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:15.603 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 109/164 2024-05-08T06:35:15.603 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-05-08T06:35:15.604 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-05-08T06:35:15.604 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-05-08T06:35:15.604 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-05-08T06:35:15.604 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-05-08T06:35:15.604 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-05-08T06:35:15.604 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-05-08T06:35:15.604 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:15.638 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gcd693 138/164 2024-05-08T06:35:15.661 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gcd693 138/164 2024-05-08T06:35:15.661 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:15.661 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T06:35:15.661 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:15.705 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 139/164 2024-05-08T06:35:15.728 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 139/164 2024-05-08T06:35:15.728 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:15.729 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T06:35:15.729 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T06:35:15.729 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T06:35:15.729 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 110/164 2024-05-08T06:35:15.801 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 110/164 2024-05-08T06:35:15.801 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:15.801 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T06:35:15.801 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:16.039 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 140/164 2024-05-08T06:35:16.063 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 140/164 2024-05-08T06:35:16.063 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:16.063 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T06:35:16.063 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T06:35:16.063 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T06:35:16.063 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:17.295 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 111/164 2024-05-08T06:35:17.322 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 111/164 2024-05-08T06:35:17.322 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:17.322 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T06:35:17.322 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T06:35:17.322 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T06:35:17.322 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:17.554 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 112/164 2024-05-08T06:35:17.577 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 112/164 2024-05-08T06:35:17.578 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:17.578 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T06:35:17.578 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T06:35:17.578 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T06:35:17.578 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:18.022 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 113/164 2024-05-08T06:35:18.045 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 113/164 2024-05-08T06:35:18.045 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:18.045 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T06:35:18.045 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T06:35:18.045 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T06:35:18.046 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:18.144 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T06:35:18.209 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T06:35:18.264 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T06:35:18.450 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T06:35:18.506 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T06:35:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T06:35:18.681 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T06:35:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T06:35:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T06:35:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T06:35:18.924 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T06:35:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 125/164 2024-05-08T06:35:19.373 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 125/164 2024-05-08T06:35:19.414 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gcd693b0b.e 126/164 2024-05-08T06:35:19.477 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gcd693b0b. 127/164 2024-05-08T06:35:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noa 128/164 2024-05-08T06:35:20.548 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noa 128/164 2024-05-08T06:35:20.582 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.n 129/164 2024-05-08T06:35:20.726 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-test-2:19.0.0-3538.gcd693b0b.el9.x86_64 141/164 2024-05-08T06:35:20.781 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 142/164 2024-05-08T06:35:20.824 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-nbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 143/164 2024-05-08T06:35:20.857 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T06:35:20.884 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T06:35:21.085 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-shell-2:19.0.0-3538.gcd693b0b.el9.x86_64 146/164 2024-05-08T06:35:21.138 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T06:35:21.168 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gcd693b0b.el9.x86_ 148/164 2024-05-08T06:35:21.198 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T06:35:21.252 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T06:35:21.548 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.n 129/164 2024-05-08T06:35:21.574 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd6 130/164 2024-05-08T06:35:21.686 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd6 130/164 2024-05-08T06:35:21.845 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gcd693b0b.el 131/164 2024-05-08T06:35:21.859 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T06:35:21.876 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 132/164 2024-05-08T06:35:21.900 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T06:35:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 132/164 2024-05-08T06:35:21.900 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:21.900 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T06:35:21.901 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T06:35:21.901 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T06:35:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:21.921 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T06:35:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 133/164 2024-05-08T06:35:21.964 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 133/164 2024-05-08T06:35:22.062 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:19.0.0-3538.gcd693b0b.el9.x86_64 134/164 2024-05-08T06:35:22.127 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 135/164 2024-05-08T06:35:22.772 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-top-2:19.0.0-3538.gcd693b0b.el9.noarch 152/164 2024-05-08T06:35:22.871 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T06:35:22.928 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T06:35:23.012 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T06:35:23.041 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T06:35:23.059 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T06:35:23.066 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T06:35:23.066 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T06:35:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 137/164 2024-05-08T06:35:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 137/164 2024-05-08T06:35:23.112 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:23.112 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T06:35:23.112 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T06:35:23.112 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T06:35:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:23.151 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T06:35:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gcd693 138/164 2024-05-08T06:35:23.178 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gcd693 138/164 2024-05-08T06:35:23.178 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:23.178 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T06:35:23.178 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:23.192 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T06:35:23.192 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T06:35:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 139/164 2024-05-08T06:35:23.246 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 139/164 2024-05-08T06:35:23.246 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:23.246 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T06:35:23.246 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T06:35:23.246 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T06:35:23.246 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:23.285 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T06:35:23.310 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T06:35:23.315 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T06:35:23.315 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T06:35:23.512 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T06:35:23.513 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T06:35:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 140/164 2024-05-08T06:35:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 140/164 2024-05-08T06:35:23.572 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T06:35:23.572 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T06:35:23.572 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T06:35:23.572 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T06:35:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:23.605 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T06:35:23.678 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T06:35:28.211 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:19.0.0-3538.gcd693b0b.el9.x86_64 141/164 2024-05-08T06:35:28.255 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 142/164 2024-05-08T06:35:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 143/164 2024-05-08T06:35:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T06:35:28.361 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T06:35:28.563 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-shell-2:19.0.0-3538.gcd693b0b.el9.x86_64 146/164 2024-05-08T06:35:28.607 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T06:35:28.638 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gcd693b0b.el9.x86_ 148/164 2024-05-08T06:35:28.667 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T06:35:28.706 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T06:35:29.250 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T06:35:29.287 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T06:35:29.307 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T06:35:29.577 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T06:35:29.577 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-2:19.0.0-3538.gcd693b0b.el9.x86_64 1/164 2024-05-08T06:35:29.577 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 2/164 2024-05-08T06:35:29.578 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 3/164 2024-05-08T06:35:29.578 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 4/164 2024-05-08T06:35:29.578 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gcd693 5/164 2024-05-08T06:35:29.578 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 6/164 2024-05-08T06:35:29.578 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 7/164 2024-05-08T06:35:29.578 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 8/164 2024-05-08T06:35:29.578 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 9/164 2024-05-08T06:35:29.578 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 10/164 2024-05-08T06:35:29.578 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 11/164 2024-05-08T06:35:29.578 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-test-2:19.0.0-3538.gcd693b0b.el9.x86_64 12/164 2024-05-08T06:35:29.578 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 13/164 2024-05-08T06:35:29.578 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gcd693b0b.el9.x86_64 14/164 2024-05-08T06:35:29.578 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gcd693b0b.el9.x86_ 15/164 2024-05-08T06:35:29.579 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 16/164 2024-05-08T06:35:29.579 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 17/164 2024-05-08T06:35:29.579 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados-devel-2:19.0.0-3538.gcd693b0b.el9.x86_6 18/164 2024-05-08T06:35:29.579 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86 19/164 2024-05-08T06:35:29.579 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 20/164 2024-05-08T06:35:29.579 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gcd693b0b.el 21/164 2024-05-08T06:35:29.579 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gcd693b0b.el9. 22/164 2024-05-08T06:35:29.579 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gcd693b0b.el9.x86_6 23/164 2024-05-08T06:35:29.579 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rados-2:19.0.0-3538.gcd693b0b.el9.x86_64 24/164 2024-05-08T06:35:29.579 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 25/164 2024-05-08T06:35:29.579 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 26/164 2024-05-08T06:35:29.579 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 27/164 2024-05-08T06:35:29.579 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 28/164 2024-05-08T06:35:29.579 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 29/164 2024-05-08T06:35:29.580 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gcd693b0b. 30/164 2024-05-08T06:35:29.580 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noa 31/164 2024-05-08T06:35:29.580 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.n 32/164 2024-05-08T06:35:29.580 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd6 33/164 2024-05-08T06:35:29.580 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gcd693b0b.el 34/164 2024-05-08T06:35:29.580 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 35/164 2024-05-08T06:35:29.580 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gcd693b0b.e 36/164 2024-05-08T06:35:29.580 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 37/164 2024-05-08T06:35:29.580 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 38/164 2024-05-08T06:35:29.580 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-top-2:19.0.0-3538.gcd693b0b.el9.noarch 39/164 2024-05-08T06:35:29.580 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T06:35:29.580 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T06:35:29.580 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T06:35:29.580 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T06:35:29.581 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T06:35:29.581 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T06:35:29.581 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T06:35:29.581 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T06:35:29.581 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T06:35:29.581 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T06:35:29.581 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T06:35:29.581 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T06:35:29.581 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T06:35:29.581 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T06:35:29.581 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T06:35:29.581 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T06:35:29.581 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T06:35:29.581 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T06:35:29.582 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T06:35:29.582 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T06:35:29.582 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T06:35:29.582 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T06:35:29.582 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T06:35:29.582 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T06:35:29.582 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T06:35:29.582 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T06:35:29.582 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T06:35:29.582 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T06:35:29.582 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T06:35:29.582 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T06:35:29.582 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T06:35:29.582 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T06:35:29.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T06:35:29.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T06:35:29.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T06:35:29.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T06:35:29.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T06:35:29.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T06:35:29.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T06:35:29.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T06:35:29.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T06:35:29.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T06:35:29.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T06:35:29.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T06:35:29.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T06:35:29.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T06:35:29.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T06:35:29.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T06:35:29.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T06:35:29.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T06:35:29.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T06:35:29.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T06:35:29.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T06:35:29.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T06:35:29.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T06:35:29.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T06:35:29.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T06:35:29.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T06:35:29.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T06:35:29.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T06:35:29.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T06:35:29.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T06:35:29.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T06:35:29.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T06:35:29.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T06:35:29.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T06:35:29.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T06:35:29.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T06:35:29.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T06:35:29.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T06:35:29.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T06:35:29.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T06:35:29.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T06:35:29.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T06:35:29.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T06:35:29.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 149/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 151/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T06:35:29.588 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T06:35:29.589 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T06:35:29.589 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T06:35:29.589 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T06:35:29.589 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T06:35:29.589 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T06:35:30.142 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-top-2:19.0.0-3538.gcd693b0b.el9.noarch 152/164 2024-05-08T06:35:30.235 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T06:35:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T06:35:30.378 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T06:35:30.413 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T06:35:30.447 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T06:35:30.447 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T06:35:30.524 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T06:35:30.557 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T06:35:30.557 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T06:35:30.624 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T06:35:30.649 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T06:35:30.655 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T06:35:30.655 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T06:35:30.844 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T06:35:30.844 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T06:35:30.920 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T06:35:30.993 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T06:35:32.221 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T06:35:32.221 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:32.222 INFO:teuthology.orchestra.run.smithi163.stdout:Upgraded: 2024-05-08T06:35:32.222 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T06:35:32.222 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T06:35:32.222 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T06:35:32.222 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T06:35:32.222 INFO:teuthology.orchestra.run.smithi163.stdout: librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.222 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.222 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T06:35:32.222 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T06:35:32.222 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-05-08T06:35:32.222 INFO:teuthology.orchestra.run.smithi163.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T06:35:32.222 INFO:teuthology.orchestra.run.smithi163.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T06:35:32.223 INFO:teuthology.orchestra.run.smithi163.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T06:35:32.223 INFO:teuthology.orchestra.run.smithi163.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T06:35:32.223 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.223 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.223 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.223 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.223 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:32.223 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.223 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.223 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.223 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:32.223 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:32.223 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:32.223 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-shell-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T06:35:32.224 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T06:35:32.225 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T06:35:32.226 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.226 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.226 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T06:35:32.226 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.226 INFO:teuthology.orchestra.run.smithi163.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T06:35:32.226 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T06:35:32.226 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T06:35:32.226 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T06:35:32.226 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T06:35:32.226 INFO:teuthology.orchestra.run.smithi163.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T06:35:32.226 INFO:teuthology.orchestra.run.smithi163.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T06:35:32.226 INFO:teuthology.orchestra.run.smithi163.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T06:35:32.226 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T06:35:32.227 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T06:35:32.227 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T06:35:32.227 INFO:teuthology.orchestra.run.smithi163.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T06:35:32.227 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T06:35:32.227 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T06:35:32.227 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T06:35:32.227 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T06:35:32.227 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T06:35:32.227 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T06:35:32.227 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T06:35:32.227 INFO:teuthology.orchestra.run.smithi163.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T06:35:32.227 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T06:35:32.227 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T06:35:32.227 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T06:35:32.228 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.228 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.228 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.228 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T06:35:32.228 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T06:35:32.228 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T06:35:32.228 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T06:35:32.228 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T06:35:32.228 INFO:teuthology.orchestra.run.smithi163.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T06:35:32.228 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T06:35:32.228 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T06:35:32.228 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T06:35:32.228 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T06:35:32.228 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T06:35:32.229 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T06:35:32.229 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T06:35:32.229 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T06:35:32.229 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T06:35:32.229 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T06:35:32.229 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T06:35:32.229 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T06:35:32.229 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T06:35:32.229 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T06:35:32.229 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T06:35:32.229 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T06:35:32.229 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T06:35:32.229 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T06:35:32.229 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T06:35:32.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T06:35:32.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T06:35:32.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T06:35:32.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T06:35:32.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T06:35:32.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T06:35:32.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T06:35:32.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T06:35:32.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T06:35:32.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T06:35:32.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T06:35:32.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T06:35:32.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T06:35:32.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T06:35:32.231 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T06:35:32.231 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.231 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.231 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T06:35:32.231 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T06:35:32.231 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T06:35:32.231 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.231 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T06:35:32.231 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T06:35:32.231 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T06:35:32.231 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T06:35:32.231 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T06:35:32.231 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T06:35:32.231 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T06:35:32.232 INFO:teuthology.orchestra.run.smithi163.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T06:35:32.233 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T06:35:32.233 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T06:35:32.233 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T06:35:32.233 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:32.233 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-05-08T06:35:32.721 DEBUG:teuthology.parallel:result is None 2024-05-08T06:35:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T06:35:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:19.0.0-3538.gcd693b0b.el9.x86_64 1/164 2024-05-08T06:35:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 2/164 2024-05-08T06:35:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 3/164 2024-05-08T06:35:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 4/164 2024-05-08T06:35:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gcd693 5/164 2024-05-08T06:35:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 6/164 2024-05-08T06:35:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 7/164 2024-05-08T06:35:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 8/164 2024-05-08T06:35:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 9/164 2024-05-08T06:35:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 10/164 2024-05-08T06:35:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 11/164 2024-05-08T06:35:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:19.0.0-3538.gcd693b0b.el9.x86_64 12/164 2024-05-08T06:35:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 13/164 2024-05-08T06:35:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gcd693b0b.el9.x86_64 14/164 2024-05-08T06:35:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gcd693b0b.el9.x86_ 15/164 2024-05-08T06:35:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 16/164 2024-05-08T06:35:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 17/164 2024-05-08T06:35:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:19.0.0-3538.gcd693b0b.el9.x86_6 18/164 2024-05-08T06:35:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86 19/164 2024-05-08T06:35:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 20/164 2024-05-08T06:35:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gcd693b0b.el 21/164 2024-05-08T06:35:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gcd693b0b.el9. 22/164 2024-05-08T06:35:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gcd693b0b.el9.x86_6 23/164 2024-05-08T06:35:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:19.0.0-3538.gcd693b0b.el9.x86_64 24/164 2024-05-08T06:35:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 25/164 2024-05-08T06:35:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 26/164 2024-05-08T06:35:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 27/164 2024-05-08T06:35:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 28/164 2024-05-08T06:35:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 29/164 2024-05-08T06:35:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gcd693b0b. 30/164 2024-05-08T06:35:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noa 31/164 2024-05-08T06:35:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.n 32/164 2024-05-08T06:35:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd6 33/164 2024-05-08T06:35:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gcd693b0b.el 34/164 2024-05-08T06:35:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 35/164 2024-05-08T06:35:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gcd693b0b.e 36/164 2024-05-08T06:35:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 37/164 2024-05-08T06:35:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 38/164 2024-05-08T06:35:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-top-2:19.0.0-3538.gcd693b0b.el9.noarch 39/164 2024-05-08T06:35:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T06:35:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T06:35:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T06:35:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T06:35:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T06:35:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T06:35:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T06:35:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T06:35:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T06:35:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T06:35:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T06:35:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T06:35:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T06:35:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T06:35:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T06:35:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T06:35:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T06:35:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T06:35:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T06:35:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T06:35:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T06:35:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T06:35:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T06:35:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T06:35:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T06:35:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T06:35:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T06:35:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T06:35:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T06:35:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T06:35:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T06:35:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T06:35:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T06:35:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T06:35:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T06:35:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T06:35:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T06:35:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T06:35:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T06:35:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T06:35:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T06:35:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T06:35:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T06:35:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T06:35:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T06:35:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T06:35:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T06:35:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T06:35:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T06:35:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T06:35:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T06:35:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T06:35:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 149/164 2024-05-08T06:35:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T06:35:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 151/164 2024-05-08T06:35:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T06:35:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T06:35:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T06:35:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T06:35:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T06:35:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T06:35:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T06:35:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T06:35:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T06:35:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T06:35:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T06:35:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T06:35:39.388 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T06:35:39.389 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:39.389 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-05-08T06:35:39.389 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T06:35:39.389 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T06:35:39.389 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T06:35:39.389 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T06:35:39.389 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.389 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.389 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T06:35:39.389 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T06:35:39.389 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T06:35:39.389 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T06:35:39.390 INFO:teuthology.orchestra.run.smithi192.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T06:35:39.390 INFO:teuthology.orchestra.run.smithi192.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T06:35:39.390 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T06:35:39.390 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.390 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.390 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.390 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.390 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:39.390 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.390 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.390 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.390 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:39.390 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:39.391 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:39.391 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:39.391 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:39.391 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.391 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.391 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:39.391 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.391 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.391 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.391 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:39.391 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:39.391 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.391 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.391 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.392 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T06:35:39.393 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T06:35:39.393 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T06:35:39.393 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T06:35:39.393 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T06:35:39.393 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.393 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.393 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T06:35:39.393 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.393 INFO:teuthology.orchestra.run.smithi192.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T06:35:39.393 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T06:35:39.393 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T06:35:39.393 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T06:35:39.393 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T06:35:39.394 INFO:teuthology.orchestra.run.smithi192.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T06:35:39.394 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T06:35:39.394 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T06:35:39.394 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T06:35:39.394 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T06:35:39.394 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T06:35:39.394 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T06:35:39.394 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T06:35:39.394 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T06:35:39.394 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T06:35:39.394 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T06:35:39.394 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T06:35:39.394 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T06:35:39.394 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T06:35:39.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T06:35:39.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T06:35:39.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T06:35:39.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T06:35:39.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T06:35:39.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T06:35:39.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T06:35:39.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T06:35:39.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T06:35:39.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T06:35:39.396 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T06:35:39.396 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T06:35:39.396 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T06:35:39.396 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T06:35:39.396 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T06:35:39.396 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T06:35:39.396 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T06:35:39.396 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T06:35:39.396 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T06:35:39.396 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T06:35:39.396 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T06:35:39.396 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T06:35:39.396 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T06:35:39.396 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T06:35:39.397 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T06:35:39.397 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T06:35:39.397 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T06:35:39.397 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T06:35:39.397 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T06:35:39.397 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T06:35:39.397 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T06:35:39.397 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T06:35:39.397 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T06:35:39.397 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T06:35:39.397 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T06:35:39.397 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T06:35:39.397 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T06:35:39.398 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T06:35:39.398 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T06:35:39.398 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T06:35:39.398 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T06:35:39.398 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T06:35:39.398 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T06:35:39.398 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T06:35:39.398 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.398 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.398 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T06:35:39.398 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T06:35:39.398 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T06:35:39.398 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T06:35:39.399 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T06:35:39.400 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.400 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.400 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T06:35:39.400 INFO:teuthology.orchestra.run.smithi192.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T06:35:39.400 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T06:35:39.400 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T06:35:39.400 INFO:teuthology.orchestra.run.smithi192.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T06:35:39.400 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T06:35:39.400 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T06:35:39.400 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T06:35:39.401 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:39.401 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T06:35:39.835 DEBUG:teuthology.parallel:result is None 2024-05-08T06:35:39.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd693b0b93278c2b128be2d6d93ae4df18c3ba1c 2024-05-08T06:35:39.986 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T06:35:40.011 INFO:teuthology.orchestra.run.smithi163.stdout:19.0.0-3538.gcd693b0b.el9 2024-05-08T06:35:40.011 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gcd693b0b.el9 2024-05-08T06:35:40.011 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gcd693b0b is installed. 2024-05-08T06:35:40.013 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd693b0b93278c2b128be2d6d93ae4df18c3ba1c 2024-05-08T06:35:40.164 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T06:35:40.187 INFO:teuthology.orchestra.run.smithi192.stdout:19.0.0-3538.gcd693b0b.el9 2024-05-08T06:35:40.187 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gcd693b0b.el9 2024-05-08T06:35:40.187 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gcd693b0b is installed. 2024-05-08T06:35:40.189 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T06:35:40.189 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:35:40.190 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T06:35:40.221 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:35:40.221 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T06:35:40.255 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T06:35:40.255 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:35:40.255 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T06:35:40.289 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T06:35:40.356 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:35:40.357 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T06:35:40.382 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T06:35:40.448 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T06:35:40.448 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:35:40.448 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T06:35:40.477 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T06:35:40.544 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:35:40.544 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T06:35:40.568 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T06:35:40.632 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T06:35:40.633 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:35:40.633 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T06:35:40.659 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T06:35:40.725 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:35:40.726 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T06:35:40.750 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T06:35:40.814 INFO:teuthology.run_tasks:Running task ceph... 2024-05-08T06:35:41.114 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-08T06:35:41.114 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 777 /var/log/ceph 2024-05-08T06:35:41.117 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /var/log/ceph 2024-05-08T06:35:41.143 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-08T06:35:41.143 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T06:35:41.185 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T06:35:41.211 INFO:tasks.ceph:Creating extra log directories... 2024-05-08T06:35:41.211 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T06:35:41.253 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T06:35:41.285 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-08T06:35:41.285 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-08T06:35:41.325 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-08T06:35:41.325 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:35:41.326 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-08T06:35:41.343 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-08T06:35:41.456 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-08T06:35:41.482 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:35:41.482 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-08T06:35:41.498 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-08T06:35:41.609 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-08T06:35:41.640 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-08T06:35:41.640 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-08T06:35:41.640 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389', 'branch': 'wip-jcollin-testing-07052024', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/mds-flush}', 'email': 'jcollin@redhat.com', 'first_in_suite': False, 'job_id': '7697389', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-jcollin-testing-07052024'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'origin-jcollin-testing', 'sha1': '6d0b29a4704de72eab52765a276866448dc1f6a5'}}, 'owner': 'scheduled_jcollin@teuthology', 'priority': 2, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5526, 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'origin-jcollin-testing', 'suite_path': '/home/teuthworker/src/github.com_joscollin_ceph_6d0b29a4704de72eab52765a276866448dc1f6a5/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/joscollin/ceph.git', 'suite_sha1': '6d0b29a4704de72eab52765a276866448dc1f6a5', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi163.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEjGUnfpYuz1hfmfIk1olfiv3+NVWGgwOJdsPkuNzIFSejmYiKRr2J8ipCci639i/dE2wiRCQvoWj1zoKeuqzoA=', 'smithi192.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKPBptJFAXy1RPfSH9ism4orrVF6u8zHozwHRD2UWZFoURNJg4XfWnLMB1IkroVAcJHKNX/FdJ7TvKDtZv80fsk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_flush']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_06:15:23', 'tube': 'smithi', 'user': 'jcollin', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-08T06:35:41.641 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T06:35:41.643 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T06:35:41.691 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T06:35:41.700 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T06:35:41.755 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:35:41.755 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T06:35:41.780 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T06:35:41.781 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_1 2024-05-08T06:35:41.837 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T06:35:41.838 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T06:35:41.838 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-08T06:35:41.838 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T06:35:41.838 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T06:35:41.838 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-05-08 06:35:24.064317165 +0000 2024-05-08T06:35:41.838 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-05-08 06:33:31.268904465 +0000 2024-05-08T06:35:41.838 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-05-08 06:33:31.268904465 +0000 2024-05-08T06:35:41.838 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-05-08 06:33:31.268904465 +0000 2024-05-08T06:35:41.839 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T06:35:41.902 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-05-08T06:35:41.902 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-05-08T06:35:41.902 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000208191 s, 2.5 MB/s 2024-05-08T06:35:41.904 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T06:35:41.961 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_2 2024-05-08T06:35:42.018 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T06:35:42.018 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T06:35:42.018 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-08T06:35:42.018 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T06:35:42.018 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T06:35:42.018 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-05-08 06:35:24.064317165 +0000 2024-05-08T06:35:42.018 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-05-08 06:33:31.601896671 +0000 2024-05-08T06:35:42.018 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-05-08 06:33:31.601896671 +0000 2024-05-08T06:35:42.018 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-05-08 06:33:31.601896671 +0000 2024-05-08T06:35:42.019 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T06:35:42.082 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-05-08T06:35:42.083 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-05-08T06:35:42.083 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000185503 s, 2.8 MB/s 2024-05-08T06:35:42.084 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T06:35:42.140 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_3 2024-05-08T06:35:42.196 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T06:35:42.197 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T06:35:42.197 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-08T06:35:42.197 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T06:35:42.197 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T06:35:42.197 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-05-08 06:35:24.065317143 +0000 2024-05-08T06:35:42.197 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-05-08 06:33:31.965888150 +0000 2024-05-08T06:35:42.197 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-05-08 06:33:31.965888150 +0000 2024-05-08T06:35:42.197 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-05-08 06:33:31.965888150 +0000 2024-05-08T06:35:42.197 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T06:35:42.260 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-05-08T06:35:42.260 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-05-08T06:35:42.260 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000195323 s, 2.6 MB/s 2024-05-08T06:35:42.261 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T06:35:42.318 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_4 2024-05-08T06:35:42.373 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T06:35:42.373 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T06:35:42.373 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-08T06:35:42.373 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T06:35:42.373 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T06:35:42.373 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-05-08 06:35:24.065317143 +0000 2024-05-08T06:35:42.374 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-05-08 06:33:32.304880216 +0000 2024-05-08T06:35:42.374 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-05-08 06:33:32.304880216 +0000 2024-05-08T06:35:42.374 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-05-08 06:33:32.304880216 +0000 2024-05-08T06:35:42.374 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T06:35:42.436 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-05-08T06:35:42.436 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-05-08T06:35:42.436 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000207976 s, 2.5 MB/s 2024-05-08T06:35:42.437 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T06:35:42.493 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T06:35:42.494 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:35:42.494 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T06:35:42.511 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T06:35:42.511 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-05-08T06:35:42.568 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T06:35:42.568 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T06:35:42.568 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-08T06:35:42.568 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T06:35:42.568 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T06:35:42.568 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-08 06:35:31.351075704 +0000 2024-05-08T06:35:42.568 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-08 06:33:34.232328826 +0000 2024-05-08T06:35:42.568 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-08 06:33:34.232328826 +0000 2024-05-08T06:35:42.568 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-08 06:33:34.232328826 +0000 2024-05-08T06:35:42.568 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T06:35:42.633 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-08T06:35:42.633 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-08T06:35:42.633 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000245801 s, 2.1 MB/s 2024-05-08T06:35:42.634 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T06:35:42.691 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-05-08T06:35:42.747 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T06:35:42.747 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T06:35:42.747 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-08T06:35:42.747 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T06:35:42.747 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T06:35:42.747 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-08 06:35:31.352075686 +0000 2024-05-08T06:35:42.747 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-08 06:33:34.577322236 +0000 2024-05-08T06:35:42.747 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-08 06:33:34.577322236 +0000 2024-05-08T06:35:42.747 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-08 06:33:34.577322236 +0000 2024-05-08T06:35:42.747 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T06:35:42.811 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-08T06:35:42.811 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-08T06:35:42.811 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000295523 s, 1.7 MB/s 2024-05-08T06:35:42.813 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T06:35:42.870 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-05-08T06:35:42.926 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T06:35:42.926 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T06:35:42.926 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-08T06:35:42.926 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T06:35:42.926 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T06:35:42.926 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-08 06:35:31.352075686 +0000 2024-05-08T06:35:42.926 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-08 06:33:34.931315473 +0000 2024-05-08T06:35:42.926 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-08 06:33:34.931315473 +0000 2024-05-08T06:35:42.927 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-08 06:33:34.931315473 +0000 2024-05-08T06:35:42.927 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T06:35:42.989 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-08T06:35:42.990 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-08T06:35:42.990 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000196405 s, 2.6 MB/s 2024-05-08T06:35:42.991 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T06:35:43.047 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-05-08T06:35:43.103 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T06:35:43.103 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T06:35:43.103 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-08T06:35:43.103 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T06:35:43.103 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T06:35:43.104 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-08 06:35:31.353075666 +0000 2024-05-08T06:35:43.104 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-08 06:33:35.261309169 +0000 2024-05-08T06:35:43.104 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-08 06:33:35.261309169 +0000 2024-05-08T06:35:43.104 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-08 06:33:35.261309169 +0000 2024-05-08T06:35:43.104 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T06:35:43.166 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-08T06:35:43.166 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-08T06:35:43.166 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000173539 s, 3.0 MB/s 2024-05-08T06:35:43.167 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T06:35:43.223 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T06:35:43.223 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi163.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T06:35:43.223 INFO:tasks.ceph:Generating config... 2024-05-08T06:35:43.249 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-08T06:35:43.249 INFO:tasks.ceph:[client] debug client = 20 2024-05-08T06:35:43.250 INFO:tasks.ceph:[client] debug ms = 1 2024-05-08T06:35:43.250 INFO:tasks.ceph:[client] pid file = 2024-05-08T06:35:43.250 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-08T06:35:43.250 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-08T06:35:43.250 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-08T06:35:43.250 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-08T06:35:43.250 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-08T06:35:43.250 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-08T06:35:43.250 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-08T06:35:43.250 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-08T06:35:43.250 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-08T06:35:43.250 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-08T06:35:43.251 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-08T06:35:43.251 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-08T06:35:43.251 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-08T06:35:43.251 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-08T06:35:43.251 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-08T06:35:43.251 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-08T06:35:43.251 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-08T06:35:43.251 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-08T06:35:43.251 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-08T06:35:43.251 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-08T06:35:43.251 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-08T06:35:43.251 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-08T06:35:43.251 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-08T06:35:43.251 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-08T06:35:43.251 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-08T06:35:43.251 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-08T06:35:43.251 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-08T06:35:43.251 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-08T06:35:43.252 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-08T06:35:43.252 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-08T06:35:43.252 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-08T06:35:43.252 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-08T06:35:43.252 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-08T06:35:43.252 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-08T06:35:43.252 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-08T06:35:43.252 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-08T06:35:43.252 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-08T06:35:43.252 INFO:tasks.ceph:Setting up mon.a... 2024-05-08T06:35:43.252 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-08T06:35:43.308 INFO:teuthology.orchestra.run.smithi163.stdout:creating /etc/ceph/ceph.keyring 2024-05-08T06:35:43.311 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-08T06:35:43.358 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T06:35:43.384 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.163'), ('mon.b', '172.21.15.192'), ('mon.c', '[v2:172.21.15.192:3301,v1:172.21.15.192:6790]')] 2024-05-08T06:35:43.384 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.163,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-08T06:35:43.385 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:35:43.385 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T06:35:43.440 DEBUG:teuthology.orchestra.run.smithi163:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.163 --add b 172.21.15.192 --addv c '[v2:172.21.15.192:3301,v1:172.21.15.192:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-08T06:35:43.528 INFO:teuthology.orchestra.run.smithi163.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-08T06:35:43.529 INFO:teuthology.orchestra.run.smithi163.stdout:monmaptool: generated fsid e639506e-cf71-49c6-ae4d-5af4601c36e0 2024-05-08T06:35:43.529 INFO:teuthology.orchestra.run.smithi163.stdout:setting min_mon_release = quincy 2024-05-08T06:35:43.529 INFO:teuthology.orchestra.run.smithi163.stdout:epoch 0 2024-05-08T06:35:43.529 INFO:teuthology.orchestra.run.smithi163.stdout:fsid e639506e-cf71-49c6-ae4d-5af4601c36e0 2024-05-08T06:35:43.529 INFO:teuthology.orchestra.run.smithi163.stdout:last_changed 2024-05-08T06:35:43.530679+0000 2024-05-08T06:35:43.529 INFO:teuthology.orchestra.run.smithi163.stdout:created 2024-05-08T06:35:43.530679+0000 2024-05-08T06:35:43.529 INFO:teuthology.orchestra.run.smithi163.stdout:min_mon_release 17 (quincy) 2024-05-08T06:35:43.529 INFO:teuthology.orchestra.run.smithi163.stdout:election_strategy: 1 2024-05-08T06:35:43.529 INFO:teuthology.orchestra.run.smithi163.stdout:0: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.a 2024-05-08T06:35:43.529 INFO:teuthology.orchestra.run.smithi163.stdout:1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.b 2024-05-08T06:35:43.529 INFO:teuthology.orchestra.run.smithi163.stdout:2: [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] mon.c 2024-05-08T06:35:43.529 INFO:teuthology.orchestra.run.smithi163.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-08T06:35:43.530 DEBUG:teuthology.orchestra.run.smithi163:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T06:35:43.546 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e639506e-cf71-49c6-ae4d-5af4601c36e0... 2024-05-08T06:35:43.546 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T06:35:43.589 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T06:35:43.631 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2024-05-08T06:35:43.632 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2024-05-08T06:35:43.632 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T06:35:43.632 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2024-05-08T06:35:43.632 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.632 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2024-05-08T06:35:43.632 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.632 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2024-05-08T06:35:43.632 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.632 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2024-05-08T06:35:43.632 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2024-05-08T06:35:43.632 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.632 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2024-05-08T06:35:43.632 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2024-05-08T06:35:43.632 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.633 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T06:35:43.633 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2024-05-08T06:35:43.633 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.633 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T06:35:43.633 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T06:35:43.633 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.633 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2024-05-08T06:35:43.633 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.633 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2024-05-08T06:35:43.633 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2024-05-08T06:35:43.633 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2024-05-08T06:35:43.633 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2024-05-08T06:35:43.633 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2024-05-08T06:35:43.633 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: mon_allow_pool_size_one = true 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: mon health detail to clog = false 2024-05-08T06:35:43.634 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.163,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]" 2024-05-08T06:35:43.635 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on skipped message = False 2024-05-08T06:35:43.635 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = e639506e-cf71-49c6-ae4d-5af4601c36e0 2024-05-08T06:35:43.635 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.635 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2024-05-08T06:35:43.635 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2024-05-08T06:35:43.635 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.635 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2024-05-08T06:35:43.635 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2024-05-08T06:35:43.635 INFO:teuthology.orchestra.run.smithi192.stdout: osd mclock profile = high_recovery_ops 2024-05-08T06:35:43.635 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.635 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2024-05-08T06:35:43.635 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2024-05-08T06:35:43.635 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug trim objects = true 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: bdev async discard = True 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: bdev enable discard = True 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore allocator = bitmap 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore block size = 96636764160 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore fsck on mount = True 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluefs = 1/20 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluestore = 1/20 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 20 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: debug rocksdb = 4/10 2024-05-08T06:35:43.637 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T06:35:43.638 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd full ratio = 0.9 2024-05-08T06:35:43.638 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd nearfull ratio = 0.8 2024-05-08T06:35:43.638 INFO:teuthology.orchestra.run.smithi192.stdout: osd failsafe full ratio = 0.95 2024-05-08T06:35:43.638 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = bluestore 2024-05-08T06:35:43.638 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2024-05-08T06:35:43.638 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.638 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2024-05-08T06:35:43.638 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-08T06:35:43.638 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2024-05-08T06:35:43.638 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2024-05-08T06:35:43.638 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2024-05-08T06:35:43.638 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2024-05-08T06:35:43.638 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2024-05-08T06:35:43.638 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.639 INFO:teuthology.orchestra.run.smithi192.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout: # 1m isn't quite enough 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout: mon_down_mkfs_grace = 2m 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_filestore_osds = false 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout: mon down mkfs grace = 300 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout: mon op complaint time = 120 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T06:35:43.640 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout: client mount timeout = 600 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = "" 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 900 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 900 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2024-05-08T06:35:43.641 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2024-05-08T06:35:43.642 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.c] 2024-05-08T06:35:43.642 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2024-05-08T06:35:43.642 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.d] 2024-05-08T06:35:43.642 INFO:teuthology.orchestra.run.smithi192.stdout:[mds] 2024-05-08T06:35:43.642 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds = 20 2024-05-08T06:35:43.642 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds balancer = 20 2024-05-08T06:35:43.642 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-08T06:35:43.642 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug frag = True 2024-05-08T06:35:43.642 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug scatterstat = True 2024-05-08T06:35:43.642 INFO:teuthology.orchestra.run.smithi192.stdout: mds op complaint time = 180 2024-05-08T06:35:43.642 INFO:teuthology.orchestra.run.smithi192.stdout: mds verify scatter = True 2024-05-08T06:35:43.642 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2024-05-08T06:35:43.642 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 900 2024-05-08T06:35:43.642 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 900 2024-05-08T06:35:43.643 INFO:teuthology.orchestra.run.smithi163.stdout:[global] 2024-05-08T06:35:43.643 INFO:teuthology.orchestra.run.smithi163.stdout: chdir = "" 2024-05-08T06:35:43.643 INFO:teuthology.orchestra.run.smithi163.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T06:35:43.643 INFO:teuthology.orchestra.run.smithi163.stdout: auth supported = cephx 2024-05-08T06:35:43.643 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.643 INFO:teuthology.orchestra.run.smithi163.stdout: filestore xattr use omap = true 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: mon clock drift allowed = 1.000 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: osd crush chooseleaf type = 0 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: auth debug = true 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on old message = true 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on bug = true 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: mon pg warn max object skew = 0 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.644 INFO:teuthology.orchestra.run.smithi163.stdout: osd pool default size = 2 2024-05-08T06:35:43.645 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.645 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd allow primary affinity = true 2024-05-08T06:35:43.645 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd allow pg remap = true 2024-05-08T06:35:43.645 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on legacy crush tunables = false 2024-05-08T06:35:43.645 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on crush straw calc version zero = false 2024-05-08T06:35:43.645 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on no sortbitwise = false 2024-05-08T06:35:43.645 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on osd down out interval zero = false 2024-05-08T06:35:43.645 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on too few osds = false 2024-05-08T06:35:43.645 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T06:35:43.645 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T06:35:43.645 INFO:teuthology.orchestra.run.smithi163.stdout: mon_allow_pool_size_one = true 2024-05-08T06:35:43.645 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.645 INFO:teuthology.orchestra.run.smithi163.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T06:35:43.645 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.646 INFO:teuthology.orchestra.run.smithi163.stdout: osd default data pool replay window = 5 2024-05-08T06:35:43.646 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.646 INFO:teuthology.orchestra.run.smithi163.stdout: mon allow pool delete = true 2024-05-08T06:35:43.646 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.646 INFO:teuthology.orchestra.run.smithi163.stdout: mon cluster log file level = debug 2024-05-08T06:35:43.646 INFO:teuthology.orchestra.run.smithi163.stdout: debug asserts on shutdown = true 2024-05-08T06:35:43.646 INFO:teuthology.orchestra.run.smithi163.stdout: mon health detail to clog = false 2024-05-08T06:35:43.646 INFO:teuthology.orchestra.run.smithi163.stdout: mon host = "172.21.15.163,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]" 2024-05-08T06:35:43.646 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on skipped message = False 2024-05-08T06:35:43.646 INFO:teuthology.orchestra.run.smithi163.stdout: fsid = e639506e-cf71-49c6-ae4d-5af4601c36e0 2024-05-08T06:35:43.646 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.646 INFO:teuthology.orchestra.run.smithi163.stdout:[osd] 2024-05-08T06:35:43.646 INFO:teuthology.orchestra.run.smithi163.stdout: osd journal size = 100 2024-05-08T06:35:43.646 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: osd scrub load threshold = 5.0 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: osd scrub max interval = 600 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: osd mclock profile = high_recovery_ops 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: osd recover clone overlap = true 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: osd recovery max chunk = 1048576 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug shutdown = true 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug op order = true 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug verify stray on activate = true 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug trim objects = true 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: osd open classes on start = true 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug pg log writeout = true 2024-05-08T06:35:43.647 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: osd deep scrub update digest min age = 30 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: osd map max advance = 10 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: journal zero on create = true 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: filestore ondisk finisher threads = 3 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: filestore apply finisher threads = 3 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: bdev debug aio = true 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug misdirected ops = true 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: bdev async discard = True 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: bdev enable discard = True 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore allocator = bitmap 2024-05-08T06:35:43.648 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore block size = 96636764160 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore fsck on mount = True 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout: debug bluefs = 1/20 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout: debug bluestore = 1/20 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout: debug osd = 20 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout: debug rocksdb = 4/10 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd full ratio = 0.9 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd nearfull ratio = 0.8 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout: osd failsafe full ratio = 0.95 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout: osd objectstore = bluestore 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout: osd op complaint time = 180 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr] 2024-05-08T06:35:43.649 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout: debug mgr = 20 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout: debug mon = 20 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout: debug auth = 20 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min pgs per osd = 4 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min bytes per osd = 10 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout: mgr/telemetry/nag = false 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout: debug client = 20 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout:[mon] 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout: debug mon = 20 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout: debug paxos = 20 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout: debug auth = 20 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout: mon data avail warn = 5 2024-05-08T06:35:43.650 INFO:teuthology.orchestra.run.smithi163.stdout: mon mgr mkfs grace = 240 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min pgs per osd = 4 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd reporter subtree level = osd 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd prime pg temp = true 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min bytes per osd = 10 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: # 1m isn't quite enough 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: mon_down_mkfs_grace = 2m 2024-05-08T06:35:43.651 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_filestore_osds = false 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout: mon down mkfs grace = 300 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout: mon op complaint time = 120 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout:[client] 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout: rgw cache enabled = true 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout: rgw enable ops log = true 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout: rgw enable usage log = true 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout: client mount timeout = 600 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout: debug client = 20 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout: pid file = "" 2024-05-08T06:35:43.652 INFO:teuthology.orchestra.run.smithi163.stdout: rados mon op timeout = 900 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout: rados osd op timeout = 900 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.a] 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.b] 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.c] 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.a] 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.c] 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.b] 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.d] 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout:[mds] 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout: debug mds = 20 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout: debug mds balancer = 20 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout: mds debug frag = True 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout: mds debug scatterstat = True 2024-05-08T06:35:43.653 INFO:teuthology.orchestra.run.smithi163.stdout: mds op complaint time = 180 2024-05-08T06:35:43.654 INFO:teuthology.orchestra.run.smithi163.stdout: mds verify scatter = True 2024-05-08T06:35:43.654 INFO:teuthology.orchestra.run.smithi163.stdout: osd op complaint time = 180 2024-05-08T06:35:43.654 INFO:teuthology.orchestra.run.smithi163.stdout: rados mon op timeout = 900 2024-05-08T06:35:43.654 INFO:teuthology.orchestra.run.smithi163.stdout: rados osd op timeout = 900 2024-05-08T06:35:43.655 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-08T06:35:43.655 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-08T06:35:43.752 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-08T06:35:43.752 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:35:43.752 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-08T06:35:43.768 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:35:43.768 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-08T06:35:43.824 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:35:43.824 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:35:43.824 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T06:35:43.824 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T06:35:43.900 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:35:43.900 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T06:35:43.955 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T06:35:43.956 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:35:43.956 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T06:35:43.956 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T06:35:43.990 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:35:43.991 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T06:35:44.047 INFO:tasks.ceph:Setting up mon nodes... 2024-05-08T06:35:44.047 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-08T06:35:44.047 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T06:35:44.100 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T06:35:44.102 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T06:35:44.159 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T06:35:44.162 INFO:tasks.ceph:Setting up mds nodes... 2024-05-08T06:35:44.162 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-08T06:35:44.213 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-08T06:35:44.216 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-08T06:35:44.241 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-08T06:35:44.332 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-08T06:35:44.335 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-08T06:35:44.359 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-08T06:35:44.410 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-08T06:35:44.413 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-08T06:35:44.437 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-08T06:35:44.528 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-08T06:35:44.531 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-08T06:35:44.554 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-08T06:35:44.555 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T06:35:44.595 INFO:teuthology.orchestra.run.smithi163.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-08T06:35:44.608 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-08T06:35:44.689 INFO:teuthology.orchestra.run.smithi163.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-08T06:35:44.701 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-08T06:35:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-08T06:35:44.754 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-08T06:35:44.835 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-08T06:35:44.848 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-08T06:35:44.848 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi163.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T06:35:44.848 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-08T06:35:44.873 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T06:35:44.873 INFO:tasks.ceph:role: osd.0 2024-05-08T06:35:44.873 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:35:44.873 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T06:35:44.945 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T06:35:44.945 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T06:35:44.945 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T06:35:44.945 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T06:35:44.946 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T06:35:44.946 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-05-08T06:35:44.946 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T06:35:44.946 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T06:35:44.946 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T06:35:44.946 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T06:35:45.459 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-05-08T06:35:45.461 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-05-08T06:35:45.461 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-08T06:35:45.570 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-08T06:35:45.601 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-08T06:35:45.666 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T06:35:45.666 INFO:tasks.ceph:role: osd.1 2024-05-08T06:35:45.666 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:35:45.667 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T06:35:45.739 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T06:35:45.739 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T06:35:45.739 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T06:35:45.739 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T06:35:45.739 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T06:35:45.739 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-05-08T06:35:45.739 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T06:35:45.739 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T06:35:45.739 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T06:35:45.740 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T06:35:46.208 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-05-08T06:35:46.210 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-05-08T06:35:46.210 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-08T06:35:46.306 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-08T06:35:46.335 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-08T06:35:46.401 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T06:35:46.401 INFO:tasks.ceph:role: osd.2 2024-05-08T06:35:46.401 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:35:46.402 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T06:35:46.474 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T06:35:46.474 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T06:35:46.474 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T06:35:46.474 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T06:35:46.475 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T06:35:46.475 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-05-08T06:35:46.475 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T06:35:46.475 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T06:35:46.475 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T06:35:46.475 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T06:35:46.961 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-05-08T06:35:46.963 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-05-08T06:35:46.963 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-08T06:35:47.081 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-08T06:35:47.111 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-08T06:35:47.177 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T06:35:47.177 INFO:tasks.ceph:role: osd.3 2024-05-08T06:35:47.177 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:35:47.177 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T06:35:47.250 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T06:35:47.250 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T06:35:47.250 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T06:35:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T06:35:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T06:35:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-05-08T06:35:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T06:35:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T06:35:47.251 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T06:35:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T06:35:47.771 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-05-08T06:35:47.773 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-05-08T06:35:47.774 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-08T06:35:47.924 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-08T06:35:47.959 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T06:35:48.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:48.063+0000 7fa71b121640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-08T06:35:48.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:48.064+0000 7fa71b121640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-08T06:35:48.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:48.064+0000 7fa71b121640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T06:35:48.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:48.064+0000 7fa71b121640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T06:35:48.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:48.064+0000 7fa71b121640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T06:35:48.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:48.064+0000 7fa71b121640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-08T06:35:50.912 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-08T06:35:50.941 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T06:35:51.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:51.032+0000 7f9bac0f0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-08T06:35:51.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:51.033+0000 7f9bac0f0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-08T06:35:51.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:51.033+0000 7f9bac0f0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T06:35:51.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:51.033+0000 7f9bac0f0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T06:35:51.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:51.033+0000 7f9bac0f0640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T06:35:51.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:51.033+0000 7f9bac0f0640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-08T06:35:53.878 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-08T06:35:53.909 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T06:35:53.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:54.000+0000 7f14eaa0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-08T06:35:54.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:54.000+0000 7f14eaa0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-08T06:35:54.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:54.000+0000 7f14eaa0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T06:35:54.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:54.000+0000 7f14eaa0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T06:35:54.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:54.000+0000 7f14eaa0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T06:35:54.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:54.001+0000 7f14eaa0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-08T06:35:56.858 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-08T06:35:56.887 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T06:35:56.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:56.980+0000 7f8e73b68640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-08T06:35:56.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:56.981+0000 7f8e73b68640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-08T06:35:56.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:56.981+0000 7f8e73b68640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T06:35:56.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:56.981+0000 7f8e73b68640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T06:35:56.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:56.981+0000 7f8e73b68640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T06:35:56.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:35:56.981+0000 7f8e73b68640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-08T06:35:59.838 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-08T06:35:59.867 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-08T06:35:59.898 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T06:35:59.898 INFO:tasks.ceph:role: osd.4 2024-05-08T06:35:59.898 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T06:35:59.898 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T06:35:59.975 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T06:35:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T06:35:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T06:35:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T06:35:59.976 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T06:35:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-08T06:35:59.976 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T06:35:59.976 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T06:35:59.976 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T06:35:59.976 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T06:36:00.480 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-08T06:36:00.482 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-08T06:36:00.482 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-08T06:36:00.605 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-08T06:36:00.637 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-08T06:36:00.702 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T06:36:00.702 INFO:tasks.ceph:role: osd.5 2024-05-08T06:36:00.702 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T06:36:00.703 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T06:36:00.774 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T06:36:00.774 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T06:36:00.774 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T06:36:00.774 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T06:36:00.774 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T06:36:00.774 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-08T06:36:00.774 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T06:36:00.774 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T06:36:00.774 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T06:36:00.775 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T06:36:01.235 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-08T06:36:01.237 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-08T06:36:01.237 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-08T06:36:01.331 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-08T06:36:01.359 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-08T06:36:01.426 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T06:36:01.426 INFO:tasks.ceph:role: osd.6 2024-05-08T06:36:01.426 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T06:36:01.426 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T06:36:01.498 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T06:36:01.498 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T06:36:01.498 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T06:36:01.499 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T06:36:01.499 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T06:36:01.499 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-08T06:36:01.499 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T06:36:01.499 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T06:36:01.499 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T06:36:01.499 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T06:36:01.981 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-08T06:36:01.983 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-08T06:36:01.983 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-08T06:36:02.109 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-08T06:36:02.141 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-08T06:36:02.206 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T06:36:02.207 INFO:tasks.ceph:role: osd.7 2024-05-08T06:36:02.207 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T06:36:02.207 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T06:36:02.279 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T06:36:02.279 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T06:36:02.279 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T06:36:02.280 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T06:36:02.280 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T06:36:02.280 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-08T06:36:02.280 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T06:36:02.280 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T06:36:02.280 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T06:36:02.280 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T06:36:02.798 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-08T06:36:02.800 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-08T06:36:02.800 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-08T06:36:02.928 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-08T06:36:02.961 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T06:36:03.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:03.055+0000 7f0222041640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-08T06:36:03.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:03.056+0000 7f0222041640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-08T06:36:03.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:03.056+0000 7f0222041640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T06:36:03.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:03.056+0000 7f0222041640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T06:36:03.056 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:03.056+0000 7f0222041640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T06:36:03.056 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:03.056+0000 7f0222041640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-08T06:36:05.904 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-08T06:36:05.934 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T06:36:06.025 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:06.026+0000 7f8b7c9b6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-08T06:36:06.026 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:06.026+0000 7f8b7c9b6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-08T06:36:06.026 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:06.026+0000 7f8b7c9b6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T06:36:06.026 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:06.026+0000 7f8b7c9b6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T06:36:06.026 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:06.026+0000 7f8b7c9b6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T06:36:06.026 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:06.026+0000 7f8b7c9b6640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-08T06:36:08.877 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-08T06:36:08.907 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T06:36:08.997 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:08.997+0000 7f930260f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-08T06:36:08.997 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:08.998+0000 7f930260f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-08T06:36:08.997 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:08.998+0000 7f930260f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T06:36:08.997 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:08.998+0000 7f930260f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T06:36:08.997 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:08.998+0000 7f930260f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T06:36:08.997 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:08.998+0000 7f930260f640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-08T06:36:11.641 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T06:36:11.644 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T06:36:11.845 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-08T06:36:11.872 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T06:36:11.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:11.959+0000 7fbce5018640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-08T06:36:11.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:11.959+0000 7fbce5018640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-08T06:36:11.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:11.959+0000 7fbce5018640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T06:36:11.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:11.959+0000 7fbce5018640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T06:36:11.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:11.959+0000 7fbce5018640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T06:36:11.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:11.959+0000 7fbce5018640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-08T06:36:14.806 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-08T06:36:14.835 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-08T06:36:14.835 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:36:14.835 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-08T06:36:14.866 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:36:14.866 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-08T06:36:14.934 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:36:14.934 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-08T06:36:15.002 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:36:15.002 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-08T06:36:15.068 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:36:15.068 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-08T06:36:15.136 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:36:15.136 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-08T06:36:15.202 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:36:15.202 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-08T06:36:15.270 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:36:15.270 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-08T06:36:15.301 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:36:15.302 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-08T06:36:15.370 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:36:15.370 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-08T06:36:15.438 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:36:15.438 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-08T06:36:15.504 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:36:15.504 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-08T06:36:15.570 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:36:15.570 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-08T06:36:15.636 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:36:15.636 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-08T06:36:15.701 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:36:15.701 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-08T06:36:15.717 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:36:15.717 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-08T06:36:15.772 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:36:15.773 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-08T06:36:15.788 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:36:15.788 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-08T06:36:15.843 INFO:tasks.ceph:Adding keys to all mons... 2024-05-08T06:36:15.843 DEBUG:teuthology.orchestra.run.smithi163:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T06:36:15.846 DEBUG:teuthology.orchestra.run.smithi192:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T06:36:15.886 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr.y] 2024-05-08T06:36:15.887 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBAHTtmkLEUBhAAsMQc6/tPdPXIPV+iJTyvNg== 2024-05-08T06:36:15.887 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.a] 2024-05-08T06:36:15.887 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBAHTtmqmfZDBAANo1hg5zAsXYQCFggPgdDrg== 2024-05-08T06:36:15.887 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.c] 2024-05-08T06:36:15.887 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBAHTtmxMbxExAAGlrRx6CvlReC6EMej743Ug== 2024-05-08T06:36:15.887 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.0] 2024-05-08T06:36:15.887 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBEHTtmqdDbAxAALZBFHAbE7N680+1LMNdFpQ== 2024-05-08T06:36:15.887 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.1] 2024-05-08T06:36:15.887 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBHHTtmvs0CAhAASCqBWYMADdQ18jnz4Em0mg== 2024-05-08T06:36:15.887 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.2] 2024-05-08T06:36:15.887 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBKHTtmj/QUABAAZo33r3p9GzUGIxrtfCx1Gg== 2024-05-08T06:36:15.887 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.3] 2024-05-08T06:36:15.887 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBMHTtmT/6EOhAA+DZAI4ARReOZuSOa8prIdA== 2024-05-08T06:36:15.887 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr.x] 2024-05-08T06:36:15.888 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBAHTtmzk6gCRAABCm0Mn6GBZL/M7B+kmzhVA== 2024-05-08T06:36:15.888 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.b] 2024-05-08T06:36:15.888 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBAHTtmUgCVGBAApW+5C4VEymZvs5rTTSRpag== 2024-05-08T06:36:15.888 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.d] 2024-05-08T06:36:15.888 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBAHTtmPqmZHxAAmJ4mxC6u5Gq1STQhAFymPQ== 2024-05-08T06:36:15.888 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.4] 2024-05-08T06:36:15.888 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBTHTtmVkRmAxAAP/TBnP/iNQzmTk7UOZEX5w== 2024-05-08T06:36:15.888 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.5] 2024-05-08T06:36:15.888 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBWHTtm8AygARAAtRm5MEdIkMZAokmbzJHqlQ== 2024-05-08T06:36:15.888 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.6] 2024-05-08T06:36:15.888 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBYHTtmuVqLOxAA7QYDS8WD1C6MJk/LxgvS1A== 2024-05-08T06:36:15.888 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.7] 2024-05-08T06:36:15.888 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBbHTtmHfg8ORAAGcnBJycy8okW7aEJMY3pug== 2024-05-08T06:36:15.889 INFO:teuthology.orchestra.run.smithi163.stdout:[client.0] 2024-05-08T06:36:15.889 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBAHTtmobegIxAApWbSIBzhwZQyiH9lGjNodA== 2024-05-08T06:36:15.889 INFO:teuthology.orchestra.run.smithi163.stdout:[client.2] 2024-05-08T06:36:15.889 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBAHTtmhewvKRAAv6ild817yPYN5MT8UQOlqg== 2024-05-08T06:36:15.889 INFO:teuthology.orchestra.run.smithi163.stdout:[client.1] 2024-05-08T06:36:15.889 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBAHTtmzf5cLBAAkDg+s+PCS5L6wwtbtpBcbw== 2024-05-08T06:36:15.889 INFO:teuthology.orchestra.run.smithi163.stdout:[client.3] 2024-05-08T06:36:15.889 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBAHTtmaITuMRAAVnocUQRC+LPUV6hDfKBblA== 2024-05-08T06:36:15.906 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.y] 2024-05-08T06:36:15.906 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBAHTtmkLEUBhAAsMQc6/tPdPXIPV+iJTyvNg== 2024-05-08T06:36:15.906 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2024-05-08T06:36:15.906 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBAHTtmqmfZDBAANo1hg5zAsXYQCFggPgdDrg== 2024-05-08T06:36:15.906 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.c] 2024-05-08T06:36:15.906 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBAHTtmxMbxExAAGlrRx6CvlReC6EMej743Ug== 2024-05-08T06:36:15.906 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.0] 2024-05-08T06:36:15.906 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBEHTtmqdDbAxAALZBFHAbE7N680+1LMNdFpQ== 2024-05-08T06:36:15.906 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.1] 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBHHTtmvs0CAhAASCqBWYMADdQ18jnz4Em0mg== 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.2] 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBKHTtmj/QUABAAZo33r3p9GzUGIxrtfCx1Gg== 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.3] 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBMHTtmT/6EOhAA+DZAI4ARReOZuSOa8prIdA== 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.x] 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBAHTtmzk6gCRAABCm0Mn6GBZL/M7B+kmzhVA== 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBAHTtmUgCVGBAApW+5C4VEymZvs5rTTSRpag== 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.d] 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBAHTtmPqmZHxAAmJ4mxC6u5Gq1STQhAFymPQ== 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.4] 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBTHTtmVkRmAxAAP/TBnP/iNQzmTk7UOZEX5w== 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.5] 2024-05-08T06:36:15.907 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBWHTtm8AygARAAtRm5MEdIkMZAokmbzJHqlQ== 2024-05-08T06:36:15.908 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.6] 2024-05-08T06:36:15.908 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBYHTtmuVqLOxAA7QYDS8WD1C6MJk/LxgvS1A== 2024-05-08T06:36:15.908 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.7] 2024-05-08T06:36:15.908 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBbHTtmHfg8ORAAGcnBJycy8okW7aEJMY3pug== 2024-05-08T06:36:15.908 INFO:teuthology.orchestra.run.smithi192.stdout:[client.0] 2024-05-08T06:36:15.908 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBAHTtmobegIxAApWbSIBzhwZQyiH9lGjNodA== 2024-05-08T06:36:15.908 INFO:teuthology.orchestra.run.smithi192.stdout:[client.2] 2024-05-08T06:36:15.908 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBAHTtmhewvKRAAv6ild817yPYN5MT8UQOlqg== 2024-05-08T06:36:15.908 INFO:teuthology.orchestra.run.smithi192.stdout:[client.1] 2024-05-08T06:36:15.908 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBAHTtmzf5cLBAAkDg+s+PCS5L6wwtbtpBcbw== 2024-05-08T06:36:15.908 INFO:teuthology.orchestra.run.smithi192.stdout:[client.3] 2024-05-08T06:36:15.908 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBAHTtmaITuMRAAVnocUQRC+LPUV6hDfKBblA== 2024-05-08T06:36:15.909 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T06:36:15.932 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T06:36:15.999 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T06:36:16.001 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T06:36:16.055 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T06:36:16.057 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T06:36:16.110 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.112 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.162 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.165 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.215 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.217 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.267 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.269 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.319 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T06:36:16.322 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T06:36:16.372 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T06:36:16.374 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T06:36:16.427 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T06:36:16.430 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T06:36:16.483 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.485 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.535 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.538 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.588 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.590 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.640 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.643 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T06:36:16.693 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T06:36:16.695 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T06:36:16.748 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T06:36:16.751 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T06:36:16.805 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T06:36:16.810 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T06:36:16.864 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T06:36:16.866 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T06:36:16.919 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-08T06:36:16.919 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-08T06:36:16.944 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T06:36:17.039 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-08T06:36:17.064 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-08T06:36:17.090 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T06:36:17.185 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-08T06:36:17.210 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-08T06:36:17.274 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T06:36:17.371 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-08T06:36:17.396 DEBUG:teuthology.orchestra.run.smithi163:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T06:36:17.398 DEBUG:teuthology.orchestra.run.smithi192:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T06:36:17.451 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-08T06:36:17.452 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T06:36:17.452 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T06:36:17.454 INFO:tasks.ceph.mon.a:Started 2024-05-08T06:36:17.454 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T06:36:17.454 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T06:36:17.494 INFO:tasks.ceph.mon.b:Started 2024-05-08T06:36:17.495 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T06:36:17.495 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T06:36:17.497 INFO:tasks.ceph.mon.c:Started 2024-05-08T06:36:17.497 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-08T06:36:17.497 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-08T06:36:17.497 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-08T06:36:17.501 INFO:tasks.ceph.mgr.y:Started 2024-05-08T06:36:17.501 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-08T06:36:17.502 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-08T06:36:17.503 INFO:tasks.ceph.mgr.x:Started 2024-05-08T06:36:17.504 INFO:tasks.ceph:Setting crush tunables to default 2024-05-08T06:36:17.504 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd crush tunables default 2024-05-08T06:36:17.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.604+0000 7fd6ebae3640 1 Processor -- start 2024-05-08T06:36:17.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.605+0000 7fd6ebae3640 1 -- start start 2024-05-08T06:36:17.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.605+0000 7fd6ebae3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e415d3a0 0x7fd6e40b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:17.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.605+0000 7fd6ebae3640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd6e4161a20 0x7fd6e40b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.605+0000 7fd6ebae3640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e40d0030 con 0x7fd6e4160eb0 2024-05-08T06:36:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.605+0000 7fd6ebae3640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd6e40b4a90 con 0x7fd6e4161a20 2024-05-08T06:36:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.605+0000 7fd6ebae3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e40b4d30 con 0x7fd6e415d3a0 2024-05-08T06:36:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.606+0000 7fd6e987f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e415d3a0 0x7fd6e40b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.606+0000 7fd6ea080640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd6e4161a20 0x7fd6e40b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.606+0000 7fd6e907e640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fd6e4160eb0 0x7fd6e4161310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:44192/0 (socket says 172.21.15.163:44192) 2024-05-08T06:36:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.606+0000 7fd6e907e640 1 -- 172.21.15.163:0/1271154277 learned_addr learned my addr 172.21.15.163:0/1271154277 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.606+0000 7fd6e987f640 1 -- 172.21.15.163:0/1271154277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e415d3a0 msgr2=0x7fd6e40b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T06:36:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.606+0000 7fd6e987f640 1 -- 172.21.15.163:0/1271154277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e415d3a0 msgr2=0x7fd6e40b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T06:36:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.606+0000 7fd6e987f640 1 --2- 172.21.15.163:0/1271154277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e415d3a0 0x7fd6e40b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T06:36:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.606+0000 7fd6e987f640 1 --2- 172.21.15.163:0/1271154277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e415d3a0 0x7fd6e40b0f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T06:36:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.606+0000 7fd6ea080640 1 -- 172.21.15.163:0/1271154277 >> v2:172.21.15.192:3300/0 conn(0x7fd6e4161a20 msgr2=0x7fd6e40b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T06:36:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.606+0000 7fd6ea080640 1 -- 172.21.15.163:0/1271154277 >> v2:172.21.15.192:3300/0 conn(0x7fd6e4161a20 msgr2=0x7fd6e40b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T06:36:17.607 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.606+0000 7fd6ea080640 1 --2- 172.21.15.163:0/1271154277 >> v2:172.21.15.192:3300/0 conn(0x7fd6e4161a20 0x7fd6e40b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T06:36:17.607 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.606+0000 7fd6ea080640 1 --2- 172.21.15.163:0/1271154277 >> v2:172.21.15.192:3300/0 conn(0x7fd6e4161a20 0x7fd6e40b5af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T06:36:17.760 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 -- 172.21.15.163:0/1271154277 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1157216996 0 0) 0x7fd6e40d0030 con 0x7fd6e4160eb0 2024-05-08T06:36:17.760 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 -- 172.21.15.163:0/1271154277 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6d0003520 con 0x7fd6e4160eb0 2024-05-08T06:36:17.760 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 -- 172.21.15.163:0/1271154277 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 404398195 0 0) 0x7fd6d0003520 con 0x7fd6e4160eb0 2024-05-08T06:36:17.760 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 -- 172.21.15.163:0/1271154277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e415d3a0 msgr2=0x7fd6e40b0f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T06:36:17.760 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 --2- 172.21.15.163:0/1271154277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e415d3a0 0x7fd6e40b0f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:17.760 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 -- 172.21.15.163:0/1271154277 >> v2:172.21.15.192:3300/0 conn(0x7fd6e4161a20 msgr2=0x7fd6e40b5af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T06:36:17.760 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 --2- 172.21.15.163:0/1271154277 >> v2:172.21.15.192:3300/0 conn(0x7fd6e4161a20 0x7fd6e40b5af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:17.760 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 -- 172.21.15.163:0/1271154277 --> v1:172.21.15.192:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e40b5010 con 0x7fd6e4160eb0 2024-05-08T06:36:17.760 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 -- 172.21.15.163:0/1271154277 <== mon.1 v1:172.21.15.192:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7fd6d40016f0 con 0x7fd6e4160eb0 2024-05-08T06:36:17.760 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 -- 172.21.15.163:0/1271154277 >> v1:172.21.15.192:6789/0 conn(0x7fd6e4160eb0 legacy=0x7fd6e4161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:17.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 --2- 172.21.15.163:0/1271154277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e415d3a0 0x7fd6d0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:17.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 --2- 172.21.15.163:0/1271154277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6d0004080 0x7fd6d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:17.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 --2- 172.21.15.163:0/1271154277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d0004d40 0x7fd6d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:17.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 -- 172.21.15.163:0/1271154277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e40b4a90 con 0x7fd6d0004080 2024-05-08T06:36:17.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 -- 172.21.15.163:0/1271154277 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e40b4d30 con 0x7fd6d0004d40 2024-05-08T06:36:17.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e887d640 1 -- 172.21.15.163:0/1271154277 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d000bc30 con 0x7fd6e415d3a0 2024-05-08T06:36:17.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.759+0000 7fd6e907e640 1 --2- 172.21.15.163:0/1271154277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6d0004080 0x7fd6d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:17.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.760+0000 7fd6ea080640 1 --2- 172.21.15.163:0/1271154277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d0004d40 0x7fd6d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:17.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.760+0000 7fd6e987f640 1 --2- 172.21.15.163:0/1271154277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e415d3a0 0x7fd6d0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:17.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.760+0000 7fd6e907e640 1 -- 172.21.15.163:0/1271154277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e415d3a0 msgr2=0x7fd6d0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:17.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.760+0000 7fd6e907e640 1 --2- 172.21.15.163:0/1271154277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e415d3a0 0x7fd6d0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:17.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.760+0000 7fd6e907e640 1 -- 172.21.15.163:0/1271154277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d0004d40 msgr2=0x7fd6d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:17.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.760+0000 7fd6e907e640 1 --2- 172.21.15.163:0/1271154277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d0004d40 0x7fd6d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:17.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.760+0000 7fd6e907e640 1 -- 172.21.15.163:0/1271154277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d0003520 con 0x7fd6d0004080 2024-05-08T06:36:17.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.761+0000 7fd6e907e640 1 --2- 172.21.15.163:0/1271154277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6d0004080 0x7fd6d0004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4007d10 tx=0x7fd6d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed04a10e61c4bef3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:17.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.762+0000 7fd6e887d640 1 -- 172.21.15.163:0/1271154277 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4007d50 con 0x7fd6d0004080 2024-05-08T06:36:17.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.762+0000 7fd6e887d640 1 -- 172.21.15.163:0/1271154277 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d4033070 con 0x7fd6d0004080 2024-05-08T06:36:17.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.762+0000 7fd6e887d640 1 -- 172.21.15.163:0/1271154277 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4005d50 con 0x7fd6d0004080 2024-05-08T06:36:17.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.762+0000 7fd6ebae3640 1 -- 172.21.15.163:0/1271154277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6d0004080 msgr2=0x7fd6d0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:17.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.762+0000 7fd6ebae3640 1 --2- 172.21.15.163:0/1271154277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6d0004080 0x7fd6d0004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4007d10 tx=0x7fd6d0005830 comp rx=0 tx=0).stop 2024-05-08T06:36:17.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.762+0000 7fd6ea080640 1 -- 172.21.15.163:0/1271154277 reap_dead start 2024-05-08T06:36:17.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.762+0000 7fd6ebae3640 1 -- 172.21.15.163:0/1271154277 shutdown_connections 2024-05-08T06:36:17.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.762+0000 7fd6ebae3640 1 -- 172.21.15.163:0/1271154277 >> 172.21.15.163:0/1271154277 conn(0x7fd6e4073af0 msgr2=0x7fd6e406c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:17.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.763+0000 7fd6ebae3640 1 -- 172.21.15.163:0/1271154277 shutdown_connections 2024-05-08T06:36:17.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.763+0000 7fd6ebae3640 1 -- 172.21.15.163:0/1271154277 wait complete. 2024-05-08T06:36:17.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.764+0000 7fd6ebae3640 1 Processor -- start 2024-05-08T06:36:17.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.764+0000 7fd6ebae3640 1 -- start start 2024-05-08T06:36:17.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6ebae3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d0004080 0x7fd6e41598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:17.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6ebae3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e414cce0 0x7fd6e415be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:17.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6ebae3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6e415c370 0x7fd6e415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:17.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6ebae3640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e415d780 con 0x7fd6e415c370 2024-05-08T06:36:17.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6ebae3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e40b55d0 con 0x7fd6d0004080 2024-05-08T06:36:17.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6ebae3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e40705a0 con 0x7fd6e414cce0 2024-05-08T06:36:17.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6ea080640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6e415c370 0x7fd6e415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:17.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6ea080640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6e415c370 0x7fd6e415eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40928/0 (socket says 172.21.15.163:40928) 2024-05-08T06:36:17.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6ea080640 1 -- 172.21.15.163:0/3341439955 learned_addr learned my addr 172.21.15.163:0/3341439955 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:17.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6e987f640 1 --2- 172.21.15.163:0/3341439955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d0004080 0x7fd6e41598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:17.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6e907e640 1 --2- 172.21.15.163:0/3341439955 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e414cce0 0x7fd6e415be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:17.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6ea080640 1 -- 172.21.15.163:0/3341439955 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e414cce0 msgr2=0x7fd6e415be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:17.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6ea080640 1 --2- 172.21.15.163:0/3341439955 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e414cce0 0x7fd6e415be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:17.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6ea080640 1 -- 172.21.15.163:0/3341439955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d0004080 msgr2=0x7fd6e41598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:17.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.765+0000 7fd6ea080640 1 --2- 172.21.15.163:0/3341439955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d0004080 0x7fd6e41598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:17.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.766+0000 7fd6ea080640 1 -- 172.21.15.163:0/3341439955 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e415e030 con 0x7fd6e415c370 2024-05-08T06:36:17.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.766+0000 7fd6ea080640 1 --2- 172.21.15.163:0/3341439955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6e415c370 0x7fd6e415eb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009610 tx=0x7fd6dc00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:17.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.766+0000 7fd6da7fc640 1 -- 172.21.15.163:0/3341439955 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc01b070 con 0x7fd6e415c370 2024-05-08T06:36:17.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.766+0000 7fd6da7fc640 1 -- 172.21.15.163:0/3341439955 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6dc003990 con 0x7fd6e415c370 2024-05-08T06:36:17.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.766+0000 7fd6da7fc640 1 -- 172.21.15.163:0/3341439955 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc003d80 con 0x7fd6e415c370 2024-05-08T06:36:17.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.766+0000 7fd6ebae3640 1 -- 172.21.15.163:0/3341439955 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e415e310 con 0x7fd6e415c370 2024-05-08T06:36:17.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.766+0000 7fd6ebae3640 1 -- 172.21.15.163:0/3341439955 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e415c7f0 con 0x7fd6e415c370 2024-05-08T06:36:17.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.767+0000 7fd6da7fc640 1 -- 172.21.15.163:0/3341439955 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd6dc004030 con 0x7fd6e415c370 2024-05-08T06:36:17.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.767+0000 7fd6da7fc640 1 -- 172.21.15.163:0/3341439955 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd6dc015a90 con 0x7fd6e415c370 2024-05-08T06:36:17.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.767+0000 7fd6ebae3640 1 -- 172.21.15.163:0/3341439955 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e406a550 con 0x7fd6e415c370 2024-05-08T06:36:17.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.771+0000 7fd6da7fc640 1 -- 172.21.15.163:0/3341439955 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd6dc015ce0 con 0x7fd6e415c370 2024-05-08T06:36:17.835 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.836+0000 7fd6ebae3640 1 -- 172.21.15.163:0/3341439955 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd6e4071270 con 0x7fd6e415c370 2024-05-08T06:36:17.837 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.838+0000 7fd6da7fc640 1 -- 172.21.15.163:0/3341439955 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fd6dc015470 con 0x7fd6e415c370 2024-05-08T06:36:17.838 INFO:teuthology.orchestra.run.smithi163.stderr:adjusted tunables profile to default 2024-05-08T06:36:17.838 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.839+0000 7fd6ebae3640 1 -- 172.21.15.163:0/3341439955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6e415c370 msgr2=0x7fd6e415eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:17.839 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.839+0000 7fd6ebae3640 1 --2- 172.21.15.163:0/3341439955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6e415c370 0x7fd6e415eb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009610 tx=0x7fd6dc00f520 comp rx=0 tx=0).stop 2024-05-08T06:36:17.839 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.839+0000 7fd6ebae3640 1 -- 172.21.15.163:0/3341439955 shutdown_connections 2024-05-08T06:36:17.839 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.839+0000 7fd6ebae3640 1 --2- 172.21.15.163:0/3341439955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6e415c370 0x7fd6e415eb10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:17.839 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.839+0000 7fd6ebae3640 1 --2- 172.21.15.163:0/3341439955 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e414cce0 0x7fd6e415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:17.839 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.839+0000 7fd6ebae3640 1 --2- 172.21.15.163:0/3341439955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d0004080 0x7fd6e41598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:17.839 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.839+0000 7fd6ebae3640 1 -- 172.21.15.163:0/3341439955 >> 172.21.15.163:0/3341439955 conn(0x7fd6e4073af0 msgr2=0x7fd6e4153680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:17.839 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.839+0000 7fd6ebae3640 1 -- 172.21.15.163:0/3341439955 shutdown_connections 2024-05-08T06:36:17.839 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:17.839+0000 7fd6ebae3640 1 -- 172.21.15.163:0/3341439955 wait complete. 2024-05-08T06:36:17.849 INFO:tasks.ceph:check_enable_crimson: False 2024-05-08T06:36:17.849 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-08T06:36:17.850 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:36:17.850 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-08T06:36:17.866 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:17.868+0000 7f5923bb8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T06:36:17.872 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:17.874+0000 7f8df9f21040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T06:36:17.876 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:36:17.876 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-08T06:36:17.941 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:36:17.942 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-08T06:36:18.007 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:36:18.007 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-08T06:36:18.024 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:18.026+0000 7f5923bb8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T06:36:18.039 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:18.040+0000 7f8df9f21040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T06:36:18.064 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:36:18.064 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-08T06:36:18.079 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:18.081+0000 7f5923bb8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T06:36:18.089 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:36:18.089 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-08T06:36:18.096 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:18.097+0000 7f8df9f21040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T06:36:18.154 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:36:18.154 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-08T06:36:18.220 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:36:18.220 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-08T06:36:18.285 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 7592d5ac-4220-44b4-9465-3cc63b2b47fd 0 2024-05-08T06:36:18.342 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:18.343+0000 7f8df9f21040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T06:36:18.355 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:18.357+0000 7f5923bb8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T06:36:18.387 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:18.388+0000 7f8df9f21040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T06:36:18.403 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:18.405+0000 7f5923bb8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T06:36:18.425 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.427+0000 7f738715f640 1 Processor -- start 2024-05-08T06:36:18.425 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.427+0000 7f738715f640 1 -- start start 2024-05-08T06:36:18.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.428+0000 7f738715f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f73800b06d0 0x7f73800b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.428+0000 7f738715f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7380162fe0 0x7f7380163430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.428+0000 7f738715f640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73800b5410 con 0x7f738015f3e0 2024-05-08T06:36:18.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.428+0000 7f738715f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f7380163970 con 0x7f7380162fe0 2024-05-08T06:36:18.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.428+0000 7f738715f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f73800b56b0 con 0x7f73800b06d0 2024-05-08T06:36:18.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.428+0000 7f737ffff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f73800b06d0 0x7f73800b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.428+0000 7f737ffff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f73800b06d0 0x7f73800b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38752/0 (socket says 172.21.15.192:38752) 2024-05-08T06:36:18.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.428+0000 7f737ffff640 1 -- 172.21.15.192:0/251391152 learned_addr learned my addr 172.21.15.192:0/251391152 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:36:18.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.428+0000 7f73856fc640 1 --2- 172.21.15.192:0/251391152 >> v2:172.21.15.163:3300/0 conn(0x7f7380162fe0 0x7f7380163430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.428+0000 7f737ffff640 1 -- 172.21.15.192:0/251391152 >> v2:172.21.15.163:3300/0 conn(0x7f7380162fe0 msgr2=0x7f7380163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:18.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.428+0000 7f737ffff640 1 --2- 172.21.15.192:0/251391152 >> v2:172.21.15.163:3300/0 conn(0x7f7380162fe0 0x7f7380163430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.428+0000 7f737ffff640 1 -- 172.21.15.192:0/251391152 >> v1:172.21.15.163:6789/0 conn(0x7f738015f3e0 legacy=0x7f73800b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:18.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.428+0000 7f737ffff640 1 -- 172.21.15.192:0/251391152 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73800b5990 con 0x7f73800b06d0 2024-05-08T06:36:18.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.429+0000 7f737ffff640 1 --2- 172.21.15.192:0/251391152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73800b06d0 0x7f73800b0b60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f736c009870 tx=0x7f736c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c8f3ad4ac83f9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:18.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.429+0000 7f737f7fe640 1 -- 172.21.15.192:0/251391152 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c037070 con 0x7f73800b06d0 2024-05-08T06:36:18.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.429+0000 7f737f7fe640 1 -- 172.21.15.192:0/251391152 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f736c032440 con 0x7f73800b06d0 2024-05-08T06:36:18.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.429+0000 7f737f7fe640 1 -- 172.21.15.192:0/251391152 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c032810 con 0x7f73800b06d0 2024-05-08T06:36:18.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.430+0000 7f738715f640 1 -- 172.21.15.192:0/251391152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73800b06d0 msgr2=0x7f73800b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:18.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.430+0000 7f738715f640 1 --2- 172.21.15.192:0/251391152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73800b06d0 0x7f73800b0b60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f736c009870 tx=0x7f736c02a210 comp rx=0 tx=0).stop 2024-05-08T06:36:18.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.430+0000 7f738715f640 1 -- 172.21.15.192:0/251391152 shutdown_connections 2024-05-08T06:36:18.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.430+0000 7f738715f640 1 --2- 172.21.15.192:0/251391152 >> v2:172.21.15.163:3300/0 conn(0x7f7380162fe0 0x7f7380163430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.430+0000 7f738715f640 1 --2- 172.21.15.192:0/251391152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73800b06d0 0x7f73800b0b60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.430+0000 7f738715f640 1 -- 172.21.15.192:0/251391152 >> 172.21.15.192:0/251391152 conn(0x7f7380073af0 msgr2=0x7f738006c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:18.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.430+0000 7f738715f640 1 -- 172.21.15.192:0/251391152 shutdown_connections 2024-05-08T06:36:18.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.430+0000 7f738715f640 1 -- 172.21.15.192:0/251391152 wait complete. 2024-05-08T06:36:18.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.431+0000 7f738715f640 1 Processor -- start 2024-05-08T06:36:18.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.431+0000 7f738715f640 1 -- start start 2024-05-08T06:36:18.432 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f738715f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73800b06d0 0x7f73801f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.432 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f738715f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f738015f3e0 0x7f738015f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.432 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f738715f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7380162fe0 0x7f73801e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.432 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f738715f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f738015f7c0 con 0x7f738015f3e0 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f738715f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f73800b5f50 con 0x7f7380162fe0 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f738715f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7380163c30 con 0x7f73800b06d0 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f7384efb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73800b06d0 0x7f73801f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f73856fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7380162fe0 0x7f73801e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f7384efb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73800b06d0 0x7f73801f8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:37530/0 (socket says 172.21.15.192:37530) 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f7384efb640 1 -- 172.21.15.192:0/246323505 learned_addr learned my addr 172.21.15.192:0/246323505 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f737ffff640 1 --2- 172.21.15.192:0/246323505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f738015f3e0 0x7f738015f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f73856fc640 1 -- 172.21.15.192:0/246323505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73800b06d0 msgr2=0x7f73801f8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f73856fc640 1 --2- 172.21.15.192:0/246323505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73800b06d0 0x7f73801f8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f73856fc640 1 -- 172.21.15.192:0/246323505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f738015f3e0 msgr2=0x7f738015f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f73856fc640 1 --2- 172.21.15.192:0/246323505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f738015f3e0 0x7f738015f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f73856fc640 1 -- 172.21.15.192:0/246323505 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738006f560 con 0x7f7380162fe0 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f7384efb640 1 --2- 172.21.15.192:0/246323505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73800b06d0 0x7f73801f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.432+0000 7f73856fc640 1 --2- 172.21.15.192:0/246323505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7380162fe0 0x7f73801e96e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f737800eba0 tx=0x7f737800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:18.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.433+0000 7f737d7fa640 1 -- 172.21.15.192:0/246323505 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378018020 con 0x7f7380162fe0 2024-05-08T06:36:18.434 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.433+0000 7f737d7fa640 1 -- 172.21.15.192:0/246323505 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f737800cd20 con 0x7f7380162fe0 2024-05-08T06:36:18.434 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.433+0000 7f737d7fa640 1 -- 172.21.15.192:0/246323505 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73780146c0 con 0x7f7380162fe0 2024-05-08T06:36:18.434 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.433+0000 7f738715f640 1 -- 172.21.15.192:0/246323505 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f738006f840 con 0x7f7380162fe0 2024-05-08T06:36:18.434 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.433+0000 7f738715f640 1 -- 172.21.15.192:0/246323505 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f738006fe00 con 0x7f7380162fe0 2024-05-08T06:36:18.434 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.433+0000 7f737d7fa640 1 -- 172.21.15.192:0/246323505 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7378005d10 con 0x7f7380162fe0 2024-05-08T06:36:18.434 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.433+0000 7f737d7fa640 1 -- 172.21.15.192:0/246323505 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f737801dd50 con 0x7f7380162fe0 2024-05-08T06:36:18.434 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.434+0000 7f738715f640 1 -- 172.21.15.192:0/246323505 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73800b0b60 con 0x7f7380162fe0 2024-05-08T06:36:18.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.438+0000 7f737d7fa640 1 -- 172.21.15.192:0/246323505 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f73800b0b60 con 0x7f7380162fe0 2024-05-08T06:36:18.508 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.510+0000 7f738715f640 1 -- 172.21.15.192:0/246323505 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7592d5ac-4220-44b4-9465-3cc63b2b47fd", "id": 0} v 0) -- 0x7f738006a550 con 0x7f7380162fe0 2024-05-08T06:36:18.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.514+0000 7f737d7fa640 1 -- 172.21.15.192:0/246323505 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7592d5ac-4220-44b4-9465-3cc63b2b47fd", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f7378022030 con 0x7f7380162fe0 2024-05-08T06:36:18.512 INFO:teuthology.orchestra.run.smithi192.stdout:0 2024-05-08T06:36:18.513 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.515+0000 7f738715f640 1 -- 172.21.15.192:0/246323505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7380162fe0 msgr2=0x7f73801e96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:18.513 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.515+0000 7f738715f640 1 --2- 172.21.15.192:0/246323505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7380162fe0 0x7f73801e96e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f737800eba0 tx=0x7f737800c490 comp rx=0 tx=0).stop 2024-05-08T06:36:18.514 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.515+0000 7f738715f640 1 -- 172.21.15.192:0/246323505 shutdown_connections 2024-05-08T06:36:18.514 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.515+0000 7f738715f640 1 --2- 172.21.15.192:0/246323505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7380162fe0 0x7f73801e96e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.514 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.515+0000 7f738715f640 1 --2- 172.21.15.192:0/246323505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f738015f3e0 0x7f738015f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.514 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.515+0000 7f738715f640 1 --2- 172.21.15.192:0/246323505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73800b06d0 0x7f73801f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.514 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.515+0000 7f738715f640 1 -- 172.21.15.192:0/246323505 >> 172.21.15.192:0/246323505 conn(0x7f7380073af0 msgr2=0x7f73800b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:18.514 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.515+0000 7f738715f640 1 -- 172.21.15.192:0/246323505 shutdown_connections 2024-05-08T06:36:18.514 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.515+0000 7f738715f640 1 -- 172.21.15.192:0/246323505 wait complete. 2024-05-08T06:36:18.524 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 181f5fb9-4a94-4fa1-b8c0-79500f415678 1 2024-05-08T06:36:18.663 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.665+0000 7fe6924cc640 1 Processor -- start 2024-05-08T06:36:18.664 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.665+0000 7fe6924cc640 1 -- start start 2024-05-08T06:36:18.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe6924cc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe68c145b10 0x7fe68c0b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe6924cc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe68c0d2270 0x7fe68c0d26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe6924cc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe68c0b5e10 con 0x7fe68c073460 2024-05-08T06:36:18.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe6924cc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe68c0d2c10 con 0x7fe68c145b10 2024-05-08T06:36:18.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe6924cc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe68c145ef0 con 0x7fe68c0d2270 2024-05-08T06:36:18.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe68b7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe68c0d2270 0x7fe68c0d26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe68bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe68c145b10 0x7fe68c0b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe68b7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe68c0d2270 0x7fe68c0d26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:37542/0 (socket says 172.21.15.192:37542) 2024-05-08T06:36:18.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe68b7fe640 1 -- 172.21.15.192:0/2084930894 learned_addr learned my addr 172.21.15.192:0/2084930894 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:36:18.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe690a69640 1 --1- 172.21.15.192:0/2084930894 >> v1:172.21.15.192:6789/0 conn(0x7fe68c073460 0x7fe68c073890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:56362/0 (socket says 172.21.15.192:56362) 2024-05-08T06:36:18.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe68b7fe640 1 -- 172.21.15.192:0/2084930894 >> v2:172.21.15.192:3300/0 conn(0x7fe68c145b10 msgr2=0x7fe68c0b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:18.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe68b7fe640 1 --2- 172.21.15.192:0/2084930894 >> v2:172.21.15.192:3300/0 conn(0x7fe68c145b10 0x7fe68c0b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe68b7fe640 1 -- 172.21.15.192:0/2084930894 >> v1:172.21.15.192:6789/0 conn(0x7fe68c073460 legacy=0x7fe68c073890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:18.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe68affd640 1 -- 172.21.15.192:0/2084930894 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2004516587 0 0) 0x7fe68c0b5e10 con 0x7fe68c073460 2024-05-08T06:36:18.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe68b7fe640 1 -- 172.21.15.192:0/2084930894 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c070200 con 0x7fe68c0d2270 2024-05-08T06:36:18.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.666+0000 7fe68bfff640 1 --2- 172.21.15.192:0/2084930894 >> v2:172.21.15.192:3300/0 conn(0x7fe68c145b10 0x7fe68c0b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:18.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.667+0000 7fe68b7fe640 1 --2- 172.21.15.192:0/2084930894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe68c0d2270 0x7fe68c0d26d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe68000b920 tx=0x7fe68002a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97f557e1f8e674be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:18.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.668+0000 7fe68affd640 1 -- 172.21.15.192:0/2084930894 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68002f710 con 0x7fe68c0d2270 2024-05-08T06:36:18.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.668+0000 7fe68affd640 1 -- 172.21.15.192:0/2084930894 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe68002f8b0 con 0x7fe68c0d2270 2024-05-08T06:36:18.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.668+0000 7fe68affd640 1 -- 172.21.15.192:0/2084930894 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68002fc80 con 0x7fe68c0d2270 2024-05-08T06:36:18.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.668+0000 7fe6924cc640 1 -- 172.21.15.192:0/2084930894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe68c0d2270 msgr2=0x7fe68c0d26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:18.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.668+0000 7fe6924cc640 1 --2- 172.21.15.192:0/2084930894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe68c0d2270 0x7fe68c0d26d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe68000b920 tx=0x7fe68002a6f0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.668+0000 7fe6924cc640 1 -- 172.21.15.192:0/2084930894 shutdown_connections 2024-05-08T06:36:18.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.668+0000 7fe6924cc640 1 --2- 172.21.15.192:0/2084930894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe68c0d2270 0x7fe68c0d26d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.668+0000 7fe6924cc640 1 --2- 172.21.15.192:0/2084930894 >> v2:172.21.15.192:3300/0 conn(0x7fe68c145b10 0x7fe68c0b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.668+0000 7fe6924cc640 1 -- 172.21.15.192:0/2084930894 >> 172.21.15.192:0/2084930894 conn(0x7fe68c06d610 msgr2=0x7fe68c06da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:18.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.668+0000 7fe6924cc640 1 -- 172.21.15.192:0/2084930894 shutdown_connections 2024-05-08T06:36:18.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.668+0000 7fe6924cc640 1 -- 172.21.15.192:0/2084930894 wait complete. 2024-05-08T06:36:18.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.669+0000 7fe6924cc640 1 Processor -- start 2024-05-08T06:36:18.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.670+0000 7fe6924cc640 1 -- start start 2024-05-08T06:36:18.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.670+0000 7fe6924cc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe68c073460 0x7fe68c15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.670+0000 7fe6924cc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe68c0d2270 0x7fe68c1f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.670+0000 7fe6924cc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe68c145b10 0x7fe68c0d0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.670+0000 7fe6924cc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c0707c0 con 0x7fe68c073460 2024-05-08T06:36:18.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.670+0000 7fe6924cc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c0710f0 con 0x7fe68c0d2270 2024-05-08T06:36:18.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.670+0000 7fe6924cc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe68c070aa0 con 0x7fe68c145b10 2024-05-08T06:36:18.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.670+0000 7fe68b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe68c0d2270 0x7fe68c1f0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.670+0000 7fe690a69640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe68c145b10 0x7fe68c0d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.670+0000 7fe68b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe68c0d2270 0x7fe68c1f0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38786/0 (socket says 172.21.15.192:38786) 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.670+0000 7fe690a69640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe68c145b10 0x7fe68c0d0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:37556/0 (socket says 172.21.15.192:37556) 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.670+0000 7fe68b7fe640 1 -- 172.21.15.192:0/2966999349 learned_addr learned my addr 172.21.15.192:0/2966999349 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.671+0000 7fe68bfff640 1 --2- 172.21.15.192:0/2966999349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe68c073460 0x7fe68c15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.671+0000 7fe690a69640 1 -- 172.21.15.192:0/2966999349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe68c0d2270 msgr2=0x7fe68c1f0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.671+0000 7fe690a69640 1 --2- 172.21.15.192:0/2966999349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe68c0d2270 0x7fe68c1f0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.671+0000 7fe690a69640 1 -- 172.21.15.192:0/2966999349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe68c073460 msgr2=0x7fe68c15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.671+0000 7fe690a69640 1 --2- 172.21.15.192:0/2966999349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe68c073460 0x7fe68c15ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.671+0000 7fe690a69640 1 -- 172.21.15.192:0/2966999349 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c1401f0 con 0x7fe68c145b10 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.671+0000 7fe690a69640 1 --2- 172.21.15.192:0/2966999349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe68c145b10 0x7fe68c0d0590 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe68400c700 tx=0x7fe68400cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.671+0000 7fe688ff9640 1 -- 172.21.15.192:0/2966999349 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684018070 con 0x7fe68c145b10 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.671+0000 7fe688ff9640 1 -- 172.21.15.192:0/2966999349 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe68400ce10 con 0x7fe68c145b10 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.671+0000 7fe6924cc640 1 -- 172.21.15.192:0/2966999349 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68c0ce400 con 0x7fe68c145b10 2024-05-08T06:36:18.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.671+0000 7fe688ff9640 1 -- 172.21.15.192:0/2966999349 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68400a880 con 0x7fe68c145b10 2024-05-08T06:36:18.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.671+0000 7fe6924cc640 1 -- 172.21.15.192:0/2966999349 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68c0ce9c0 con 0x7fe68c145b10 2024-05-08T06:36:18.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.671+0000 7fe688ff9640 1 -- 172.21.15.192:0/2966999349 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe68401b070 con 0x7fe68c145b10 2024-05-08T06:36:18.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.671+0000 7fe688ff9640 1 -- 172.21.15.192:0/2966999349 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fe68401d440 con 0x7fe68c145b10 2024-05-08T06:36:18.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.672+0000 7fe6924cc640 1 -- 172.21.15.192:0/2966999349 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe68c06a550 con 0x7fe68c145b10 2024-05-08T06:36:18.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.674+0000 7fe688ff9640 1 -- 172.21.15.192:0/2966999349 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe6840070d0 con 0x7fe68c145b10 2024-05-08T06:36:18.745 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.746+0000 7fe6924cc640 1 -- 172.21.15.192:0/2966999349 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "181f5fb9-4a94-4fa1-b8c0-79500f415678", "id": 1} v 0) -- 0x7fe68c0b6640 con 0x7fe68c145b10 2024-05-08T06:36:18.748 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.750+0000 7fe688ff9640 1 -- 172.21.15.192:0/2966999349 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "181f5fb9-4a94-4fa1-b8c0-79500f415678", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fe68400aa20 con 0x7fe68c145b10 2024-05-08T06:36:18.749 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-05-08T06:36:18.750 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.751+0000 7fe6924cc640 1 -- 172.21.15.192:0/2966999349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe68c145b10 msgr2=0x7fe68c0d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:18.750 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.751+0000 7fe6924cc640 1 --2- 172.21.15.192:0/2966999349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe68c145b10 0x7fe68c0d0590 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe68400c700 tx=0x7fe68400cbc0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.750 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.751+0000 7fe6924cc640 1 -- 172.21.15.192:0/2966999349 shutdown_connections 2024-05-08T06:36:18.750 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.751+0000 7fe6924cc640 1 --2- 172.21.15.192:0/2966999349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe68c145b10 0x7fe68c0d0590 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.750 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.751+0000 7fe6924cc640 1 --2- 172.21.15.192:0/2966999349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe68c0d2270 0x7fe68c1f0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.750 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.751+0000 7fe6924cc640 1 --2- 172.21.15.192:0/2966999349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe68c073460 0x7fe68c15ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.750 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.751+0000 7fe6924cc640 1 -- 172.21.15.192:0/2966999349 >> 172.21.15.192:0/2966999349 conn(0x7fe68c06d610 msgr2=0x7fe68c144db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:18.750 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.751+0000 7fe6924cc640 1 -- 172.21.15.192:0/2966999349 shutdown_connections 2024-05-08T06:36:18.750 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.751+0000 7fe6924cc640 1 -- 172.21.15.192:0/2966999349 wait complete. 2024-05-08T06:36:18.761 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 5a16ce86-0f45-447c-9b9f-50cfc689a750 2 2024-05-08T06:36:18.861 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.863+0000 7fa35aee1640 1 Processor -- start 2024-05-08T06:36:18.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.863+0000 7fa35aee1640 1 -- start start 2024-05-08T06:36:18.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.864+0000 7fa35aee1640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa35415d3a0 0x7fa3540b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.864+0000 7fa35aee1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa354160eb0 0x7fa354161310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.864+0000 7fa35aee1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa354161ce0 con 0x7fa354161850 2024-05-08T06:36:18.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.864+0000 7fa35aee1640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa3540d0030 con 0x7fa35415d3a0 2024-05-08T06:36:18.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.864+0000 7fa35aee1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa3540b4a90 con 0x7fa354160eb0 2024-05-08T06:36:18.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.864+0000 7fa358c7d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa35415d3a0 0x7fa3540b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.864+0000 7fa358c7d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa35415d3a0 0x7fa3540b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38792/0 (socket says 172.21.15.192:38792) 2024-05-08T06:36:18.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.864+0000 7fa353fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa354160eb0 0x7fa354161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.864+0000 7fa358c7d640 1 -- 172.21.15.192:0/2523970994 learned_addr learned my addr 172.21.15.192:0/2523970994 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:36:18.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.864+0000 7fa358c7d640 1 -- 172.21.15.192:0/2523970994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa354160eb0 msgr2=0x7fa354161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:18.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.864+0000 7fa358c7d640 1 --2- 172.21.15.192:0/2523970994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa354160eb0 0x7fa354161310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.864+0000 7fa358c7d640 1 -- 172.21.15.192:0/2523970994 >> v1:172.21.15.163:6789/0 conn(0x7fa354161850 legacy=0x7fa3540ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:18.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.864+0000 7fa358c7d640 1 -- 172.21.15.192:0/2523970994 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3540b4d30 con 0x7fa35415d3a0 2024-05-08T06:36:18.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.864+0000 7fa353fff640 1 --2- 172.21.15.192:0/2523970994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa354160eb0 0x7fa354161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:18.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.865+0000 7fa358c7d640 1 --2- 172.21.15.192:0/2523970994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35415d3a0 0x7fa3540b0f50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa348009870 tx=0x7fa34802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8585666ddbfbf8a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:18.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.865+0000 7fa3537fe640 1 -- 172.21.15.192:0/2523970994 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348037070 con 0x7fa35415d3a0 2024-05-08T06:36:18.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.865+0000 7fa3537fe640 1 -- 172.21.15.192:0/2523970994 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa348032440 con 0x7fa35415d3a0 2024-05-08T06:36:18.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.865+0000 7fa3537fe640 1 -- 172.21.15.192:0/2523970994 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348032810 con 0x7fa35415d3a0 2024-05-08T06:36:18.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.866+0000 7fa35aee1640 1 -- 172.21.15.192:0/2523970994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35415d3a0 msgr2=0x7fa3540b0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:18.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.866+0000 7fa35aee1640 1 --2- 172.21.15.192:0/2523970994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35415d3a0 0x7fa3540b0f50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa348009870 tx=0x7fa34802a130 comp rx=0 tx=0).stop 2024-05-08T06:36:18.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.866+0000 7fa35aee1640 1 -- 172.21.15.192:0/2523970994 shutdown_connections 2024-05-08T06:36:18.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.866+0000 7fa35aee1640 1 --2- 172.21.15.192:0/2523970994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa354160eb0 0x7fa354161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.866+0000 7fa35aee1640 1 --2- 172.21.15.192:0/2523970994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35415d3a0 0x7fa3540b0f50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.866+0000 7fa35aee1640 1 -- 172.21.15.192:0/2523970994 >> 172.21.15.192:0/2523970994 conn(0x7fa354073af0 msgr2=0x7fa35406c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:18.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.866+0000 7fa35aee1640 1 -- 172.21.15.192:0/2523970994 shutdown_connections 2024-05-08T06:36:18.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.866+0000 7fa35aee1640 1 -- 172.21.15.192:0/2523970994 wait complete. 2024-05-08T06:36:18.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.867+0000 7fa35aee1640 1 Processor -- start 2024-05-08T06:36:18.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.867+0000 7fa35aee1640 1 -- start start 2024-05-08T06:36:18.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa35aee1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35415d3a0 0x7fa3541592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa35aee1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa354160eb0 0x7fa35414de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa35aee1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa354161850 0x7fa3541484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:18.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa35aee1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa3540b55d0 con 0x7fa354161850 2024-05-08T06:36:18.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa35aee1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa3540b52f0 con 0x7fa35415d3a0 2024-05-08T06:36:18.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa35aee1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa35415d780 con 0x7fa354160eb0 2024-05-08T06:36:18.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa358c7d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35415d3a0 0x7fa3541592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa353fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa354160eb0 0x7fa35414de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa358c7d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35415d3a0 0x7fa3541592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38808/0 (socket says 172.21.15.192:38808) 2024-05-08T06:36:18.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa358c7d640 1 -- 172.21.15.192:0/3433088357 learned_addr learned my addr 172.21.15.192:0/3433088357 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:36:18.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa35947e640 1 --2- 172.21.15.192:0/3433088357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa354161850 0x7fa3541484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:18.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa358c7d640 1 -- 172.21.15.192:0/3433088357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa354160eb0 msgr2=0x7fa35414de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:18.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa358c7d640 1 --2- 172.21.15.192:0/3433088357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa354160eb0 0x7fa35414de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa358c7d640 1 -- 172.21.15.192:0/3433088357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa354161850 msgr2=0x7fa3541484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:18.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa358c7d640 1 --2- 172.21.15.192:0/3433088357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa354161850 0x7fa3541484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa358c7d640 1 -- 172.21.15.192:0/3433088357 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35414ee10 con 0x7fa35415d3a0 2024-05-08T06:36:18.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.868+0000 7fa358c7d640 1 --2- 172.21.15.192:0/3433088357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35415d3a0 0x7fa3541592e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa34802a170 tx=0x7fa3480046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:18.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.869+0000 7fa3517fa640 1 -- 172.21.15.192:0/3433088357 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348037040 con 0x7fa35415d3a0 2024-05-08T06:36:18.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.869+0000 7fa3517fa640 1 -- 172.21.15.192:0/3433088357 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa348042070 con 0x7fa35415d3a0 2024-05-08T06:36:18.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.869+0000 7fa3517fa640 1 -- 172.21.15.192:0/3433088357 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348030880 con 0x7fa35415d3a0 2024-05-08T06:36:18.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.869+0000 7fa35aee1640 1 -- 172.21.15.192:0/3433088357 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35415ae40 con 0x7fa35415d3a0 2024-05-08T06:36:18.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.869+0000 7fa35aee1640 1 -- 172.21.15.192:0/3433088357 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35414e850 con 0x7fa35415d3a0 2024-05-08T06:36:18.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.869+0000 7fa3517fa640 1 -- 172.21.15.192:0/3433088357 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa348030a20 con 0x7fa35415d3a0 2024-05-08T06:36:18.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.869+0000 7fa3517fa640 1 -- 172.21.15.192:0/3433088357 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fa34803b430 con 0x7fa35415d3a0 2024-05-08T06:36:18.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.870+0000 7fa35aee1640 1 -- 172.21.15.192:0/3433088357 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35415b010 con 0x7fa35415d3a0 2024-05-08T06:36:18.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.872+0000 7fa3517fa640 1 -- 172.21.15.192:0/3433088357 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa354161ce0 con 0x7fa35415d3a0 2024-05-08T06:36:18.943 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.944+0000 7fa35aee1640 1 -- 172.21.15.192:0/3433088357 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5a16ce86-0f45-447c-9b9f-50cfc689a750", "id": 2} v 0) -- 0x7fa35415ac60 con 0x7fa35415d3a0 2024-05-08T06:36:18.948 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.949+0000 7fa3517fa640 1 -- 172.21.15.192:0/3433088357 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a16ce86-0f45-447c-9b9f-50cfc689a750", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fa35415ac60 con 0x7fa35415d3a0 2024-05-08T06:36:18.948 INFO:teuthology.orchestra.run.smithi192.stdout:2 2024-05-08T06:36:18.949 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.950+0000 7fa35aee1640 1 -- 172.21.15.192:0/3433088357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35415d3a0 msgr2=0x7fa3541592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:18.949 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.950+0000 7fa35aee1640 1 --2- 172.21.15.192:0/3433088357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35415d3a0 0x7fa3541592e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa34802a170 tx=0x7fa3480046c0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.949 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.950+0000 7fa35aee1640 1 -- 172.21.15.192:0/3433088357 shutdown_connections 2024-05-08T06:36:18.949 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.950+0000 7fa35aee1640 1 --2- 172.21.15.192:0/3433088357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa354161850 0x7fa3541484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.949 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.950+0000 7fa35aee1640 1 --2- 172.21.15.192:0/3433088357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa354160eb0 0x7fa35414de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.949 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.950+0000 7fa35aee1640 1 --2- 172.21.15.192:0/3433088357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35415d3a0 0x7fa3541592e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:18.949 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.951+0000 7fa35aee1640 1 -- 172.21.15.192:0/3433088357 >> 172.21.15.192:0/3433088357 conn(0x7fa354073af0 msgr2=0x7fa35406c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:18.949 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.951+0000 7fa35aee1640 1 -- 172.21.15.192:0/3433088357 shutdown_connections 2024-05-08T06:36:18.949 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:18.951+0000 7fa35aee1640 1 -- 172.21.15.192:0/3433088357 wait complete. 2024-05-08T06:36:18.959 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new c0b95302-e771-4514-9347-56dad6b5b6b7 3 2024-05-08T06:36:19.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.060+0000 7fc79ffc1640 1 Processor -- start 2024-05-08T06:36:19.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.060+0000 7fc79ffc1640 1 -- start start 2024-05-08T06:36:19.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.061+0000 7fc79ffc1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc798070d80 0x7fc798160e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.061+0000 7fc79ffc1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc79815d7a0 con 0x7fc798140100 2024-05-08T06:36:19.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.061+0000 7fc79ffc1640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc798161b70 con 0x7fc798161340 2024-05-08T06:36:19.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.061+0000 7fc79ffc1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc79815da60 con 0x7fc798070d80 2024-05-08T06:36:19.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.061+0000 7fc79d55c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc798070d80 0x7fc798160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.061+0000 7fc79e55e640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fc798161340 0x7fc7980d19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:56374/0 (socket says 172.21.15.192:56374) 2024-05-08T06:36:19.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.061+0000 7fc79e55e640 1 -- 172.21.15.192:0/1645444827 learned_addr learned my addr 172.21.15.192:0/1645444827 (peer_addr_for_me v1:172.21.15.192:0/0) 2024-05-08T06:36:19.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.061+0000 7fc79cd5b640 1 -- 172.21.15.192:0/1645444827 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1999278614 0 0) 0x7fc798161b70 con 0x7fc798161340 2024-05-08T06:36:19.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.061+0000 7fc79d55c640 1 -- 172.21.15.192:0/1645444827 >> v1:172.21.15.192:6789/0 conn(0x7fc798161340 legacy=0x7fc7980d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.061+0000 7fc79d55c640 1 -- 172.21.15.192:0/1645444827 >> v1:172.21.15.163:6789/0 conn(0x7fc798140100 legacy=0x7fc798070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.061+0000 7fc79d55c640 1 -- 172.21.15.192:0/1645444827 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79815dd40 con 0x7fc798070d80 2024-05-08T06:36:19.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.062+0000 7fc79d55c640 1 --2- 172.21.15.192:0/1645444827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc798070d80 0x7fc798160e00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc7880097d0 tx=0x7fc78802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39ee4a3daee0720 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:19.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.062+0000 7fc79cd5b640 1 -- 172.21.15.192:0/1645444827 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788037070 con 0x7fc798070d80 2024-05-08T06:36:19.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.062+0000 7fc79cd5b640 1 -- 172.21.15.192:0/1645444827 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc788032440 con 0x7fc798070d80 2024-05-08T06:36:19.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.062+0000 7fc79cd5b640 1 -- 172.21.15.192:0/1645444827 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788032810 con 0x7fc798070d80 2024-05-08T06:36:19.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.063+0000 7fc79ffc1640 1 -- 172.21.15.192:0/1645444827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc798070d80 msgr2=0x7fc798160e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.063+0000 7fc79ffc1640 1 --2- 172.21.15.192:0/1645444827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc798070d80 0x7fc798160e00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc7880097d0 tx=0x7fc78802a180 comp rx=0 tx=0).stop 2024-05-08T06:36:19.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.063+0000 7fc79ffc1640 1 -- 172.21.15.192:0/1645444827 shutdown_connections 2024-05-08T06:36:19.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.063+0000 7fc79ffc1640 1 --2- 172.21.15.192:0/1645444827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc798070d80 0x7fc798160e00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.063+0000 7fc79ffc1640 1 -- 172.21.15.192:0/1645444827 >> 172.21.15.192:0/1645444827 conn(0x7fc798072a60 msgr2=0x7fc79806c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:19.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.063+0000 7fc79ffc1640 1 -- 172.21.15.192:0/1645444827 shutdown_connections 2024-05-08T06:36:19.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.063+0000 7fc79ffc1640 1 -- 172.21.15.192:0/1645444827 wait complete. 2024-05-08T06:36:19.065 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.064+0000 7fc79ffc1640 1 Processor -- start 2024-05-08T06:36:19.065 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.064+0000 7fc79ffc1640 1 -- start start 2024-05-08T06:36:19.065 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79ffc1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc798070d80 0x7fc798154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.065 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79ffc1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc798140100 0x7fc7981521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.065 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79ffc1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc798161340 0x7fc798152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.065 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79ffc1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc7981617a0 con 0x7fc798070d80 2024-05-08T06:36:19.065 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79ffc1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7981404e0 con 0x7fc798161340 2024-05-08T06:36:19.065 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79ffc1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc798161920 con 0x7fc798140100 2024-05-08T06:36:19.065 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79e55e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc798161340 0x7fc798152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.066 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79d55c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc798140100 0x7fc7981521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.066 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79d55c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc798140100 0x7fc7981521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:37590/0 (socket says 172.21.15.192:37590) 2024-05-08T06:36:19.066 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79e55e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc798161340 0x7fc798152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38810/0 (socket says 172.21.15.192:38810) 2024-05-08T06:36:19.066 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79d55c640 1 -- 172.21.15.192:0/4118616710 learned_addr learned my addr 172.21.15.192:0/4118616710 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:36:19.066 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79dd5d640 1 --2- 172.21.15.192:0/4118616710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc798070d80 0x7fc798154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.066 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79d55c640 1 -- 172.21.15.192:0/4118616710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc798161340 msgr2=0x7fc798152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.066 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79d55c640 1 --2- 172.21.15.192:0/4118616710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc798161340 0x7fc798152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.066 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79d55c640 1 -- 172.21.15.192:0/4118616710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc798070d80 msgr2=0x7fc798154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.066 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79d55c640 1 --2- 172.21.15.192:0/4118616710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc798070d80 0x7fc798154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.066 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79d55c640 1 -- 172.21.15.192:0/4118616710 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7980b3ee0 con 0x7fc798140100 2024-05-08T06:36:19.066 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79d55c640 1 --2- 172.21.15.192:0/4118616710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc798140100 0x7fc7981521f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc788035040 tx=0x7fc7880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:19.066 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc78e7fc640 1 -- 172.21.15.192:0/4118616710 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788037050 con 0x7fc798140100 2024-05-08T06:36:19.066 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc78e7fc640 1 -- 172.21.15.192:0/4118616710 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc788038040 con 0x7fc798140100 2024-05-08T06:36:19.067 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79ffc1640 1 -- 172.21.15.192:0/4118616710 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7981578d0 con 0x7fc798140100 2024-05-08T06:36:19.067 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc78e7fc640 1 -- 172.21.15.192:0/4118616710 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788004490 con 0x7fc798140100 2024-05-08T06:36:19.067 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.065+0000 7fc79ffc1640 1 -- 172.21.15.192:0/4118616710 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc798153040 con 0x7fc798140100 2024-05-08T06:36:19.067 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.066+0000 7fc78e7fc640 1 -- 172.21.15.192:0/4118616710 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc788030900 con 0x7fc798140100 2024-05-08T06:36:19.067 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.066+0000 7fc78e7fc640 1 -- 172.21.15.192:0/4118616710 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fc78803e450 con 0x7fc798140100 2024-05-08T06:36:19.067 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.066+0000 7fc79ffc1640 1 -- 172.21.15.192:0/4118616710 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc79806a550 con 0x7fc798140100 2024-05-08T06:36:19.067 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.068+0000 7fc78e7fc640 1 -- 172.21.15.192:0/4118616710 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc7980b3ee0 con 0x7fc798140100 2024-05-08T06:36:19.139 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.141+0000 7fc79ffc1640 1 -- 172.21.15.192:0/4118616710 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c0b95302-e771-4514-9347-56dad6b5b6b7", "id": 3} v 0) -- 0x7fc798068c50 con 0x7fc798140100 2024-05-08T06:36:19.145 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.146+0000 7fc78e7fc640 1 -- 172.21.15.192:0/4118616710 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c0b95302-e771-4514-9347-56dad6b5b6b7", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fc788030d30 con 0x7fc798140100 2024-05-08T06:36:19.145 INFO:teuthology.orchestra.run.smithi192.stdout:3 2024-05-08T06:36:19.146 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.147+0000 7fc79ffc1640 1 -- 172.21.15.192:0/4118616710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc798140100 msgr2=0x7fc7981521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.146 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.147+0000 7fc79ffc1640 1 --2- 172.21.15.192:0/4118616710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc798140100 0x7fc7981521f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc788035040 tx=0x7fc7880046c0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.146 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.148+0000 7fc79ffc1640 1 -- 172.21.15.192:0/4118616710 shutdown_connections 2024-05-08T06:36:19.146 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.148+0000 7fc79ffc1640 1 --2- 172.21.15.192:0/4118616710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc798161340 0x7fc798152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.146 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.148+0000 7fc79ffc1640 1 --2- 172.21.15.192:0/4118616710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc798140100 0x7fc7981521f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.147 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.148+0000 7fc79ffc1640 1 --2- 172.21.15.192:0/4118616710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc798070d80 0x7fc798154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.147 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.148+0000 7fc79ffc1640 1 -- 172.21.15.192:0/4118616710 >> 172.21.15.192:0/4118616710 conn(0x7fc798072a60 msgr2=0x7fc79815e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:19.147 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.148+0000 7fc79ffc1640 1 -- 172.21.15.192:0/4118616710 shutdown_connections 2024-05-08T06:36:19.147 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.148+0000 7fc79ffc1640 1 -- 172.21.15.192:0/4118616710 wait complete. 2024-05-08T06:36:19.154 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:19.155+0000 7f8df9f21040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T06:36:19.156 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 78c7a01f-b6b3-411a-8f8d-a82da908ad37 4 2024-05-08T06:36:19.178 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:19.179+0000 7f5923bb8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T06:36:19.240 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:19.241+0000 7f8df9f21040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T06:36:19.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.257+0000 7f024270a640 1 Processor -- start 2024-05-08T06:36:19.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.257+0000 7f024270a640 1 -- start start 2024-05-08T06:36:19.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.258+0000 7f024270a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f023c0d2270 0x7f023c0d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.258+0000 7f024270a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f023c070200 con 0x7f023c0afb90 2024-05-08T06:36:19.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.258+0000 7f024270a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f023c0704e0 con 0x7f023c145b10 2024-05-08T06:36:19.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.258+0000 7f024270a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f023c0d2c20 con 0x7f023c0d2270 2024-05-08T06:36:19.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.258+0000 7f023b7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f023c0d2270 0x7f023c0d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.258+0000 7f023bfff640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f023c145b10 0x7f023c0b58d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:56390/0 (socket says 172.21.15.192:56390) 2024-05-08T06:36:19.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.258+0000 7f023bfff640 1 -- 172.21.15.192:0/2891589088 learned_addr learned my addr 172.21.15.192:0/2891589088 (peer_addr_for_me v1:172.21.15.192:0/0) 2024-05-08T06:36:19.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.258+0000 7f023b7fe640 1 -- 172.21.15.192:0/2891589088 >> v1:172.21.15.192:6789/0 conn(0x7f023c145b10 legacy=0x7f023c0b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.258+0000 7f023b7fe640 1 -- 172.21.15.192:0/2891589088 >> v1:172.21.15.163:6789/0 conn(0x7f023c0afb90 legacy=0x7f023c0aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.258+0000 7f023b7fe640 1 -- 172.21.15.192:0/2891589088 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023c0707c0 con 0x7f023c0d2270 2024-05-08T06:36:19.258 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.259+0000 7f023b7fe640 1 --2- 172.21.15.192:0/2891589088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c0d2270 0x7f023c0d26e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02300097a0 tx=0x7f023002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=698150bcabb38074 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:19.259 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.259+0000 7f023affd640 1 -- 172.21.15.192:0/2891589088 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230037070 con 0x7f023c0d2270 2024-05-08T06:36:19.259 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.259+0000 7f023affd640 1 -- 172.21.15.192:0/2891589088 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0230032440 con 0x7f023c0d2270 2024-05-08T06:36:19.259 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.259+0000 7f023affd640 1 -- 172.21.15.192:0/2891589088 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230032810 con 0x7f023c0d2270 2024-05-08T06:36:19.259 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.260+0000 7f024270a640 1 -- 172.21.15.192:0/2891589088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c0d2270 msgr2=0x7f023c0d26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.259 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.260+0000 7f024270a640 1 --2- 172.21.15.192:0/2891589088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c0d2270 0x7f023c0d26e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02300097a0 tx=0x7f023002a230 comp rx=0 tx=0).stop 2024-05-08T06:36:19.259 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.260+0000 7f024270a640 1 -- 172.21.15.192:0/2891589088 shutdown_connections 2024-05-08T06:36:19.259 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.260+0000 7f024270a640 1 --2- 172.21.15.192:0/2891589088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c0d2270 0x7f023c0d26e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.259 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.260+0000 7f024270a640 1 -- 172.21.15.192:0/2891589088 >> 172.21.15.192:0/2891589088 conn(0x7f023c06d610 msgr2=0x7f023c06da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:19.259 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.260+0000 7f024270a640 1 -- 172.21.15.192:0/2891589088 shutdown_connections 2024-05-08T06:36:19.259 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.260+0000 7f024270a640 1 -- 172.21.15.192:0/2891589088 wait complete. 2024-05-08T06:36:19.261 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.261+0000 7f024270a640 1 Processor -- start 2024-05-08T06:36:19.262 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.261+0000 7f024270a640 1 -- start start 2024-05-08T06:36:19.262 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f024270a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c0afb90 0x7f023c15bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.262 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f024270a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f023c0d2270 0x7f023c15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.262 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f024270a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f023c145b10 0x7f023c0d0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.262 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f024270a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f023c0710f0 con 0x7f023c0d2270 2024-05-08T06:36:19.262 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f024270a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f023c145ef0 con 0x7f023c0afb90 2024-05-08T06:36:19.262 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f024270a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f023c070d80 con 0x7f023c145b10 2024-05-08T06:36:19.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f023bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c0afb90 0x7f023c15bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f0240ca7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f023c145b10 0x7f023c0d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f023bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c0afb90 0x7f023c15bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38830/0 (socket says 172.21.15.192:38830) 2024-05-08T06:36:19.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f023bfff640 1 -- 172.21.15.192:0/916156090 learned_addr learned my addr 172.21.15.192:0/916156090 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:36:19.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f0240ca7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f023c145b10 0x7f023c0d0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:37602/0 (socket says 172.21.15.192:37602) 2024-05-08T06:36:19.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f023b7fe640 1 --2- 172.21.15.192:0/916156090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f023c0d2270 0x7f023c15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f0240ca7640 1 -- 172.21.15.192:0/916156090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c0afb90 msgr2=0x7f023c15bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f0240ca7640 1 --2- 172.21.15.192:0/916156090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c0afb90 0x7f023c15bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f0240ca7640 1 -- 172.21.15.192:0/916156090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f023c0d2270 msgr2=0x7f023c15cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f0240ca7640 1 --2- 172.21.15.192:0/916156090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f023c0d2270 0x7f023c15cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f0240ca7640 1 -- 172.21.15.192:0/916156090 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023c15e090 con 0x7f023c145b10 2024-05-08T06:36:19.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.262+0000 7f023bfff640 1 --2- 172.21.15.192:0/916156090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c0afb90 0x7f023c15bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:19.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.263+0000 7f0240ca7640 1 --2- 172.21.15.192:0/916156090 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f023c145b10 0x7f023c0d0190 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f023400c730 tx=0x7f023400cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:19.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.263+0000 7f0238ff9640 1 -- 172.21.15.192:0/916156090 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0234003b70 con 0x7f023c145b10 2024-05-08T06:36:19.264 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.263+0000 7f0238ff9640 1 -- 172.21.15.192:0/916156090 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0234003d10 con 0x7f023c145b10 2024-05-08T06:36:19.264 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.263+0000 7f0238ff9640 1 -- 172.21.15.192:0/916156090 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02340052f0 con 0x7f023c145b10 2024-05-08T06:36:19.264 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.263+0000 7f024270a640 1 -- 172.21.15.192:0/916156090 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023c15d7f0 con 0x7f023c145b10 2024-05-08T06:36:19.264 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.263+0000 7f024270a640 1 -- 172.21.15.192:0/916156090 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f023c15ddb0 con 0x7f023c145b10 2024-05-08T06:36:19.264 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.263+0000 7f0238ff9640 1 -- 172.21.15.192:0/916156090 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f02340056a0 con 0x7f023c145b10 2024-05-08T06:36:19.264 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.263+0000 7f0238ff9640 1 -- 172.21.15.192:0/916156090 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f0234012b10 con 0x7f023c145b10 2024-05-08T06:36:19.264 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.264+0000 7f024270a640 1 -- 172.21.15.192:0/916156090 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f023c06a550 con 0x7f023c145b10 2024-05-08T06:36:19.264 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.266+0000 7f0238ff9640 1 -- 172.21.15.192:0/916156090 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0234012d60 con 0x7f023c145b10 2024-05-08T06:36:19.271 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:19.273+0000 7f5923bb8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T06:36:19.282 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:19.284+0000 7f8df9f21040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T06:36:19.316 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:19.317+0000 7f5923bb8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T06:36:19.338 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.339+0000 7f024270a640 1 -- 172.21.15.192:0/916156090 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "78c7a01f-b6b3-411a-8f8d-a82da908ad37", "id": 4} v 0) -- 0x7f023c0cdf30 con 0x7f023c145b10 2024-05-08T06:36:19.341 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.343+0000 7f0238ff9640 1 -- 172.21.15.192:0/916156090 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "78c7a01f-b6b3-411a-8f8d-a82da908ad37", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f02340126d0 con 0x7f023c145b10 2024-05-08T06:36:19.341 INFO:teuthology.orchestra.run.smithi192.stdout:4 2024-05-08T06:36:19.342 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.344+0000 7f024270a640 1 -- 172.21.15.192:0/916156090 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f023c145b10 msgr2=0x7f023c0d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.342 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.344+0000 7f024270a640 1 --2- 172.21.15.192:0/916156090 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f023c145b10 0x7f023c0d0190 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f023400c730 tx=0x7f023400cbf0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.344+0000 7f024270a640 1 -- 172.21.15.192:0/916156090 shutdown_connections 2024-05-08T06:36:19.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.344+0000 7f024270a640 1 --2- 172.21.15.192:0/916156090 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f023c145b10 0x7f023c0d0190 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.344+0000 7f024270a640 1 --2- 172.21.15.192:0/916156090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f023c0d2270 0x7f023c15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.344+0000 7f024270a640 1 --2- 172.21.15.192:0/916156090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c0afb90 0x7f023c15bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.344+0000 7f024270a640 1 -- 172.21.15.192:0/916156090 >> 172.21.15.192:0/916156090 conn(0x7f023c06d610 msgr2=0x7f023c06d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:19.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.344+0000 7f024270a640 1 -- 172.21.15.192:0/916156090 shutdown_connections 2024-05-08T06:36:19.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.344+0000 7f024270a640 1 -- 172.21.15.192:0/916156090 wait complete. 2024-05-08T06:36:19.353 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 5729f316-c353-468c-929b-44184879d177 5 2024-05-08T06:36:19.448 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:19.449+0000 7f8df9f21040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T06:36:19.492 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:19.494+0000 7f5923bb8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T06:36:19.494 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.495+0000 7f006bac8640 1 Processor -- start 2024-05-08T06:36:19.494 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.496+0000 7f006bac8640 1 -- start start 2024-05-08T06:36:19.495 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.496+0000 7f006bac8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00641563d0 0x7f0064160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.496+0000 7f006bac8640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f00640b0db0 0x7f0064073d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.496+0000 7f006bac8640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00640b12b0 con 0x7f00640d2180 2024-05-08T06:36:19.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.497+0000 7f006bac8640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f006415fe70 con 0x7f00640b0db0 2024-05-08T06:36:19.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.497+0000 7f006bac8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f00640b1590 con 0x7f00641563d0 2024-05-08T06:36:19.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.497+0000 7f0069063640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f00640b0db0 0x7f0064073d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.497+0000 7f0069864640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00641563d0 0x7f0064160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.497+0000 7f006a065640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f00640d2180 0x7f0064162fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:56392/0 (socket says 172.21.15.192:56392) 2024-05-08T06:36:19.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.497+0000 7f006a065640 1 -- 172.21.15.192:0/3566570528 learned_addr learned my addr 172.21.15.192:0/3566570528 (peer_addr_for_me v1:172.21.15.192:0/0) 2024-05-08T06:36:19.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.497+0000 7f0069063640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f00640b0db0 0x7f0064073d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38842/0 (socket says 172.21.15.192:38842) 2024-05-08T06:36:19.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.497+0000 7f0069063640 1 -- 172.21.15.192:0/3566570528 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00641563d0 msgr2=0x7f0064160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:19.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.497+0000 7f0069063640 1 --2- 172.21.15.192:0/3566570528 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00641563d0 0x7f0064160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.497+0000 7f0069063640 1 -- 172.21.15.192:0/3566570528 >> v1:172.21.15.192:6789/0 conn(0x7f00640d2180 legacy=0x7f0064162fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.497+0000 7f0069063640 1 -- 172.21.15.192:0/3566570528 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006415fb00 con 0x7f00640b0db0 2024-05-08T06:36:19.498 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.498+0000 7f0069063640 1 --2- 172.21.15.192:0/3566570528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00640b0db0 0x7f0064073d40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f00540097a0 tx=0x7f005402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe475bc8049cff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:19.498 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.498+0000 7f0068862640 1 -- 172.21.15.192:0/3566570528 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054037070 con 0x7f00640b0db0 2024-05-08T06:36:19.498 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.498+0000 7f0068862640 1 -- 172.21.15.192:0/3566570528 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0054032440 con 0x7f00640b0db0 2024-05-08T06:36:19.498 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.498+0000 7f0068862640 1 -- 172.21.15.192:0/3566570528 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054032810 con 0x7f00640b0db0 2024-05-08T06:36:19.498 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.498+0000 7f006bac8640 1 -- 172.21.15.192:0/3566570528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00640b0db0 msgr2=0x7f0064073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.498 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.498+0000 7f006bac8640 1 --2- 172.21.15.192:0/3566570528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00640b0db0 0x7f0064073d40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f00540097a0 tx=0x7f005402a180 comp rx=0 tx=0).stop 2024-05-08T06:36:19.498 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.499+0000 7f006bac8640 1 -- 172.21.15.192:0/3566570528 shutdown_connections 2024-05-08T06:36:19.498 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.499+0000 7f006bac8640 1 --2- 172.21.15.192:0/3566570528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00640b0db0 0x7f0064073d40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.498 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.499+0000 7f006bac8640 1 --2- 172.21.15.192:0/3566570528 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00641563d0 0x7f0064160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.498 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.499+0000 7f006bac8640 1 -- 172.21.15.192:0/3566570528 >> 172.21.15.192:0/3566570528 conn(0x7f006406d610 msgr2=0x7f006406da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:19.498 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.499+0000 7f006bac8640 1 -- 172.21.15.192:0/3566570528 shutdown_connections 2024-05-08T06:36:19.498 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.499+0000 7f006bac8640 1 -- 172.21.15.192:0/3566570528 wait complete. 2024-05-08T06:36:19.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.500+0000 7f006bac8640 1 Processor -- start 2024-05-08T06:36:19.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.500+0000 7f006bac8640 1 -- start start 2024-05-08T06:36:19.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f006bac8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00640b0db0 0x7f00640b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f006bac8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00640d2180 0x7f0064143910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f006bac8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00641563d0 0x7f0064143e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f006bac8640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0064074280 con 0x7f00641563d0 2024-05-08T06:36:19.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f006bac8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0064160840 con 0x7f00640d2180 2024-05-08T06:36:19.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f006bac8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f00641567b0 con 0x7f00640b0db0 2024-05-08T06:36:19.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f0069864640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00640b0db0 0x7f00640b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f0069864640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00640b0db0 0x7f00640b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:37626/0 (socket says 172.21.15.192:37626) 2024-05-08T06:36:19.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f0069864640 1 -- 172.21.15.192:0/1399112499 learned_addr learned my addr 172.21.15.192:0/1399112499 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:36:19.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f0069063640 1 --2- 172.21.15.192:0/1399112499 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00640d2180 0x7f0064143910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f006a065640 1 --2- 172.21.15.192:0/1399112499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00641563d0 0x7f0064143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f0069864640 1 -- 172.21.15.192:0/1399112499 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00640d2180 msgr2=0x7f0064143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f0069864640 1 --2- 172.21.15.192:0/1399112499 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00640d2180 0x7f0064143910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f0069864640 1 -- 172.21.15.192:0/1399112499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00641563d0 msgr2=0x7f0064143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f0069864640 1 --2- 172.21.15.192:0/1399112499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00641563d0 0x7f0064143e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f0069864640 1 -- 172.21.15.192:0/1399112499 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00640b24e0 con 0x7f00640b0db0 2024-05-08T06:36:19.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f0069063640 1 --2- 172.21.15.192:0/1399112499 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00640d2180 0x7f0064143910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:19.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f006a065640 1 --2- 172.21.15.192:0/1399112499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00641563d0 0x7f0064143e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:19.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f0069864640 1 --2- 172.21.15.192:0/1399112499 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00640b0db0 0x7f00640b2fa0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f006000e9e0 tx=0x7f006000eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:19.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f005a7fc640 1 -- 172.21.15.192:0/1399112499 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060017070 con 0x7f00640b0db0 2024-05-08T06:36:19.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f005a7fc640 1 -- 172.21.15.192:0/1399112499 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f006000c9d0 con 0x7f00640b0db0 2024-05-08T06:36:19.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f005a7fc640 1 -- 172.21.15.192:0/1399112499 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006000cdc0 con 0x7f00640b0db0 2024-05-08T06:36:19.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f006bac8640 1 -- 172.21.15.192:0/1399112499 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0064150d10 con 0x7f00640b0db0 2024-05-08T06:36:19.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.501+0000 7f006bac8640 1 -- 172.21.15.192:0/1399112499 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f00640b27a0 con 0x7f00640b0db0 2024-05-08T06:36:19.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.502+0000 7f005a7fc640 1 -- 172.21.15.192:0/1399112499 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f006001a070 con 0x7f00640b0db0 2024-05-08T06:36:19.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.502+0000 7f005a7fc640 1 -- 172.21.15.192:0/1399112499 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f00600066b0 con 0x7f00640b0db0 2024-05-08T06:36:19.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.502+0000 7f006bac8640 1 -- 172.21.15.192:0/1399112499 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f006406a550 con 0x7f00640b0db0 2024-05-08T06:36:19.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.504+0000 7f005a7fc640 1 -- 172.21.15.192:0/1399112499 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0060005990 con 0x7f00640b0db0 2024-05-08T06:36:19.507 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:19.509+0000 7f8df9f21040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T06:36:19.541 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:19.543+0000 7f5923bb8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T06:36:19.560 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:19.561+0000 7f8df9f21040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T06:36:19.573 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.575+0000 7f006bac8640 1 -- 172.21.15.192:0/1399112499 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5729f316-c353-468c-929b-44184879d177", "id": 5} v 0) -- 0x7f006406db90 con 0x7f00640b0db0 2024-05-08T06:36:19.579 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.581+0000 7f005a7fc640 1 -- 172.21.15.192:0/1399112499 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5729f316-c353-468c-929b-44184879d177", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f0060005c40 con 0x7f00640b0db0 2024-05-08T06:36:19.579 INFO:teuthology.orchestra.run.smithi192.stdout:5 2024-05-08T06:36:19.581 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.582+0000 7f006bac8640 1 -- 172.21.15.192:0/1399112499 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00640b0db0 msgr2=0x7f00640b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.581 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.582+0000 7f006bac8640 1 --2- 172.21.15.192:0/1399112499 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00640b0db0 0x7f00640b2fa0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f006000e9e0 tx=0x7f006000eea0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.581 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.582+0000 7f006bac8640 1 -- 172.21.15.192:0/1399112499 shutdown_connections 2024-05-08T06:36:19.581 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.582+0000 7f006bac8640 1 --2- 172.21.15.192:0/1399112499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00641563d0 0x7f0064143e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.581 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.582+0000 7f006bac8640 1 --2- 172.21.15.192:0/1399112499 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00640d2180 0x7f0064143910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.581 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.582+0000 7f006bac8640 1 --2- 172.21.15.192:0/1399112499 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00640b0db0 0x7f00640b2fa0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.581 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.582+0000 7f006bac8640 1 -- 172.21.15.192:0/1399112499 >> 172.21.15.192:0/1399112499 conn(0x7f006406d610 msgr2=0x7f006406fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:19.581 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.582+0000 7f006bac8640 1 -- 172.21.15.192:0/1399112499 shutdown_connections 2024-05-08T06:36:19.581 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.582+0000 7f006bac8640 1 -- 172.21.15.192:0/1399112499 wait complete. 2024-05-08T06:36:19.591 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 63e9d47f-50fe-4ac6-b7bf-de0256fdccfe 6 2024-05-08T06:36:19.596 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:19.598+0000 7f5923bb8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T06:36:19.674 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:19.674+0000 7f8df9f21040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T06:36:19.690 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.691+0000 7f297390a640 1 Processor -- start 2024-05-08T06:36:19.690 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.692+0000 7f297390a640 1 -- start start 2024-05-08T06:36:19.691 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.692+0000 7f297390a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f296c145b10 0x7f296c0b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.692+0000 7f297390a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296c0d2270 0x7f296c0d26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.692+0000 7f297390a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f296c073460 0x7f296c073890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.692+0000 7f297390a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f296c0d2c10 con 0x7f296c145b10 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.692+0000 7f297390a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f296c145ef0 con 0x7f296c073460 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.692+0000 7f297390a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f296c0710f0 con 0x7f296c0d2270 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.693+0000 7f2970ea5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296c0d2270 0x7f296c0d26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.693+0000 7f2971ea7640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f296c073460 0x7f296c073890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.693+0000 7f2970ea5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296c0d2270 0x7f296c0d26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:37636/0 (socket says 172.21.15.192:37636) 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.693+0000 7f2971ea7640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f296c073460 0x7f296c073890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38858/0 (socket says 172.21.15.192:38858) 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.693+0000 7f2970ea5640 1 -- 172.21.15.192:0/3002640435 learned_addr learned my addr 172.21.15.192:0/3002640435 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.693+0000 7f29716a6640 1 --2- 172.21.15.192:0/3002640435 >> v2:172.21.15.163:3300/0 conn(0x7f296c145b10 0x7f296c0b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.693+0000 7f2970ea5640 1 -- 172.21.15.192:0/3002640435 >> v2:172.21.15.192:3300/0 conn(0x7f296c073460 msgr2=0x7f296c073890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.693+0000 7f2970ea5640 1 --2- 172.21.15.192:0/3002640435 >> v2:172.21.15.192:3300/0 conn(0x7f296c073460 0x7f296c073890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.693+0000 7f2970ea5640 1 -- 172.21.15.192:0/3002640435 >> v2:172.21.15.163:3300/0 conn(0x7f296c145b10 msgr2=0x7f296c0b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:19.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.693+0000 7f2970ea5640 1 --2- 172.21.15.192:0/3002640435 >> v2:172.21.15.163:3300/0 conn(0x7f296c145b10 0x7f296c0b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.693 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.693+0000 7f2970ea5640 1 -- 172.21.15.192:0/3002640435 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c0b5e10 con 0x7f296c0d2270 2024-05-08T06:36:19.693 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.693+0000 7f2971ea7640 1 --2- 172.21.15.192:0/3002640435 >> v2:172.21.15.192:3300/0 conn(0x7f296c073460 0x7f296c073890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:19.694 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.694+0000 7f2970ea5640 1 --2- 172.21.15.192:0/3002640435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296c0d2270 0x7f296c0d26d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f296c0b7590 tx=0x7f296002a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c75fbe1b1ff821cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:19.694 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.694+0000 7f295ffff640 1 -- 172.21.15.192:0/3002640435 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960037070 con 0x7f296c0d2270 2024-05-08T06:36:19.694 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.694+0000 7f295ffff640 1 -- 172.21.15.192:0/3002640435 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2960032440 con 0x7f296c0d2270 2024-05-08T06:36:19.694 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.694+0000 7f295ffff640 1 -- 172.21.15.192:0/3002640435 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960032810 con 0x7f296c0d2270 2024-05-08T06:36:19.694 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.694+0000 7f297390a640 1 -- 172.21.15.192:0/3002640435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296c0d2270 msgr2=0x7f296c0d26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.694 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.694+0000 7f297390a640 1 --2- 172.21.15.192:0/3002640435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296c0d2270 0x7f296c0d26d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f296c0b7590 tx=0x7f296002a1e0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.695 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.694+0000 7f297390a640 1 -- 172.21.15.192:0/3002640435 shutdown_connections 2024-05-08T06:36:19.695 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.694+0000 7f297390a640 1 --2- 172.21.15.192:0/3002640435 >> v2:172.21.15.192:3300/0 conn(0x7f296c073460 0x7f296c073890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.695 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.694+0000 7f297390a640 1 --2- 172.21.15.192:0/3002640435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296c0d2270 0x7f296c0d26d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.695 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.694+0000 7f297390a640 1 --2- 172.21.15.192:0/3002640435 >> v2:172.21.15.163:3300/0 conn(0x7f296c145b10 0x7f296c0b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.695 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.694+0000 7f297390a640 1 -- 172.21.15.192:0/3002640435 >> 172.21.15.192:0/3002640435 conn(0x7f296c06d610 msgr2=0x7f296c06da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:19.695 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.694+0000 7f297390a640 1 -- 172.21.15.192:0/3002640435 shutdown_connections 2024-05-08T06:36:19.695 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.695+0000 7f297390a640 1 -- 172.21.15.192:0/3002640435 wait complete. 2024-05-08T06:36:19.695 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.696+0000 7f297390a640 1 Processor -- start 2024-05-08T06:36:19.695 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.696+0000 7f297390a640 1 -- start start 2024-05-08T06:36:19.697 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.696+0000 7f297390a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296c073460 0x7f296c139af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.697 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.696+0000 7f297390a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f296c0d2270 0x7f296c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.697 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.696+0000 7f297390a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f296c145b10 0x7f296c138920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.697 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.696+0000 7f297390a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f296c0707c0 con 0x7f296c145b10 2024-05-08T06:36:19.697 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.696+0000 7f297390a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f296c0704e0 con 0x7f296c0d2270 2024-05-08T06:36:19.698 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.696+0000 7f297390a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f296c070aa0 con 0x7f296c073460 2024-05-08T06:36:19.698 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.696+0000 7f29716a6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296c073460 0x7f296c139af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.698 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.696+0000 7f2970ea5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f296c0d2270 0x7f296c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.698 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.696+0000 7f29716a6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296c073460 0x7f296c139af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:37638/0 (socket says 172.21.15.192:37638) 2024-05-08T06:36:19.698 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.696+0000 7f29716a6640 1 -- 172.21.15.192:0/2270306317 learned_addr learned my addr 172.21.15.192:0/2270306317 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:36:19.698 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.696+0000 7f2971ea7640 1 --2- 172.21.15.192:0/2270306317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f296c145b10 0x7f296c138920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.698 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.697+0000 7f2970ea5640 1 -- 172.21.15.192:0/2270306317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296c073460 msgr2=0x7f296c139af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.698 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.697+0000 7f2970ea5640 1 --2- 172.21.15.192:0/2270306317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296c073460 0x7f296c139af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.698 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.697+0000 7f2970ea5640 1 -- 172.21.15.192:0/2270306317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f296c145b10 msgr2=0x7f296c138920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.698 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.697+0000 7f2970ea5640 1 --2- 172.21.15.192:0/2270306317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f296c145b10 0x7f296c138920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.698 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.697+0000 7f2970ea5640 1 -- 172.21.15.192:0/2270306317 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c0d0ce0 con 0x7f296c0d2270 2024-05-08T06:36:19.698 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.697+0000 7f29716a6640 1 --2- 172.21.15.192:0/2270306317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296c073460 0x7f296c139af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:19.698 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.697+0000 7f2970ea5640 1 --2- 172.21.15.192:0/2270306317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f296c0d2270 0x7f296c0ce240 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2960031000 tx=0x7f29600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:19.698 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.697+0000 7f295dffb640 1 -- 172.21.15.192:0/2270306317 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960041020 con 0x7f296c0d2270 2024-05-08T06:36:19.699 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.697+0000 7f295dffb640 1 -- 172.21.15.192:0/2270306317 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296003d660 con 0x7f296c0d2270 2024-05-08T06:36:19.699 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.697+0000 7f295dffb640 1 -- 172.21.15.192:0/2270306317 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960037040 con 0x7f296c0d2270 2024-05-08T06:36:19.699 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.697+0000 7f297390a640 1 -- 172.21.15.192:0/2270306317 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f296c1396d0 con 0x7f296c0d2270 2024-05-08T06:36:19.699 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.697+0000 7f297390a640 1 -- 172.21.15.192:0/2270306317 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f296c1393f0 con 0x7f296c0d2270 2024-05-08T06:36:19.699 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.698+0000 7f295dffb640 1 -- 172.21.15.192:0/2270306317 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2960044020 con 0x7f296c0d2270 2024-05-08T06:36:19.699 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.698+0000 7f295dffb640 1 -- 172.21.15.192:0/2270306317 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f2960046ae0 con 0x7f296c0d2270 2024-05-08T06:36:19.699 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.698+0000 7f297390a640 1 -- 172.21.15.192:0/2270306317 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f296c06a550 con 0x7f296c0d2270 2024-05-08T06:36:19.699 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.700+0000 7f295dffb640 1 -- 172.21.15.192:0/2270306317 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f296004b030 con 0x7f296c0d2270 2024-05-08T06:36:19.715 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:19.717+0000 7f5923bb8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T06:36:19.771 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.772+0000 7f297390a640 1 -- 172.21.15.192:0/2270306317 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "63e9d47f-50fe-4ac6-b7bf-de0256fdccfe", "id": 6} v 0) -- 0x7f296c06f4b0 con 0x7f296c0d2270 2024-05-08T06:36:19.776 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.778+0000 7f295dffb640 1 -- 172.21.15.192:0/2270306317 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "63e9d47f-50fe-4ac6-b7bf-de0256fdccfe", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f2960046400 con 0x7f296c0d2270 2024-05-08T06:36:19.777 INFO:teuthology.orchestra.run.smithi192.stdout:6 2024-05-08T06:36:19.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.779+0000 7f297390a640 1 -- 172.21.15.192:0/2270306317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f296c0d2270 msgr2=0x7f296c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.779+0000 7f297390a640 1 --2- 172.21.15.192:0/2270306317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f296c0d2270 0x7f296c0ce240 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2960031000 tx=0x7f29600046c0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.779+0000 7f297390a640 1 -- 172.21.15.192:0/2270306317 shutdown_connections 2024-05-08T06:36:19.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.779+0000 7f297390a640 1 --2- 172.21.15.192:0/2270306317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f296c145b10 0x7f296c138920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.779+0000 7f297390a640 1 --2- 172.21.15.192:0/2270306317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f296c0d2270 0x7f296c0ce240 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.779+0000 7f297390a640 1 --2- 172.21.15.192:0/2270306317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296c073460 0x7f296c139af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.779+0000 7f297390a640 1 -- 172.21.15.192:0/2270306317 >> 172.21.15.192:0/2270306317 conn(0x7f296c06d610 msgr2=0x7f296c0b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:19.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.779+0000 7f297390a640 1 -- 172.21.15.192:0/2270306317 shutdown_connections 2024-05-08T06:36:19.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.779+0000 7f297390a640 1 -- 172.21.15.192:0/2270306317 wait complete. 2024-05-08T06:36:19.788 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new b1615978-384e-436f-a343-60cc5bd9d058 7 2024-05-08T06:36:19.928 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.930+0000 7fe10170a640 1 Processor -- start 2024-05-08T06:36:19.928 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.930+0000 7fe10170a640 1 -- start start 2024-05-08T06:36:19.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.930+0000 7fe10170a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0fc15f310 0x7fe0fc0bad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.930+0000 7fe10170a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0fc0b8b50 con 0x7fe0fc0d3520 2024-05-08T06:36:19.930 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.930+0000 7fe10170a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0fc0b8e30 con 0x7fe0fc0d3980 2024-05-08T06:36:19.930 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.930+0000 7fe10170a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe0fc0b9760 con 0x7fe0fc15f310 2024-05-08T06:36:19.930 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.931+0000 7fe0faffd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0fc15f310 0x7fe0fc0bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.930 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.931+0000 7fe0faffd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0fc15f310 0x7fe0fc0bad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:37652/0 (socket says 172.21.15.192:37652) 2024-05-08T06:36:19.930 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.931+0000 7fe0fb7fe640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fe0fc0d3980 0x7fe0fc0d21f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:56404/0 (socket says 172.21.15.192:56404) 2024-05-08T06:36:19.930 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.931+0000 7fe0faffd640 1 -- 172.21.15.192:0/859547188 learned_addr learned my addr 172.21.15.192:0/859547188 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:36:19.930 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.931+0000 7fe0f9ffb640 1 -- 172.21.15.192:0/859547188 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 639441359 0 0) 0x7fe0fc0b8e30 con 0x7fe0fc0d3980 2024-05-08T06:36:19.930 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.931+0000 7fe0f9ffb640 1 -- 172.21.15.192:0/859547188 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0e4003520 con 0x7fe0fc0d3980 2024-05-08T06:36:19.930 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.931+0000 7fe0faffd640 1 -- 172.21.15.192:0/859547188 >> v1:172.21.15.192:6789/0 conn(0x7fe0fc0d3980 legacy=0x7fe0fc0d21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.930 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.931+0000 7fe0faffd640 1 -- 172.21.15.192:0/859547188 >> v1:172.21.15.163:6789/0 conn(0x7fe0fc0d3520 legacy=0x7fe0fc0b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.930 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.931+0000 7fe0faffd640 1 -- 172.21.15.192:0/859547188 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc0b9110 con 0x7fe0fc15f310 2024-05-08T06:36:19.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.932+0000 7fe0faffd640 1 --2- 172.21.15.192:0/859547188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0fc15f310 0x7fe0fc0bad90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0098a0 tx=0x7fe0ec02a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d4fafea600f91a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:19.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.932+0000 7fe0f9ffb640 1 -- 172.21.15.192:0/859547188 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec037070 con 0x7fe0fc15f310 2024-05-08T06:36:19.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.932+0000 7fe0f9ffb640 1 -- 172.21.15.192:0/859547188 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0ec032640 con 0x7fe0fc15f310 2024-05-08T06:36:19.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.932+0000 7fe0f9ffb640 1 -- 172.21.15.192:0/859547188 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec032a10 con 0x7fe0fc15f310 2024-05-08T06:36:19.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.932+0000 7fe10170a640 1 -- 172.21.15.192:0/859547188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0fc15f310 msgr2=0x7fe0fc0bad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.932+0000 7fe10170a640 1 --2- 172.21.15.192:0/859547188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0fc15f310 0x7fe0fc0bad90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0098a0 tx=0x7fe0ec02a370 comp rx=0 tx=0).stop 2024-05-08T06:36:19.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.932+0000 7fe10170a640 1 -- 172.21.15.192:0/859547188 shutdown_connections 2024-05-08T06:36:19.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.932+0000 7fe10170a640 1 --2- 172.21.15.192:0/859547188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0fc15f310 0x7fe0fc0bad90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.932+0000 7fe10170a640 1 -- 172.21.15.192:0/859547188 >> 172.21.15.192:0/859547188 conn(0x7fe0fc073e70 msgr2=0x7fe0fc074280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:19.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.933+0000 7fe10170a640 1 -- 172.21.15.192:0/859547188 shutdown_connections 2024-05-08T06:36:19.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.933+0000 7fe10170a640 1 -- 172.21.15.192:0/859547188 wait complete. 2024-05-08T06:36:19.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.934+0000 7fe10170a640 1 Processor -- start 2024-05-08T06:36:19.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.934+0000 7fe10170a640 1 -- start start 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe10170a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0fc0d3520 0x7fe0fc162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe10170a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0fc0d3980 0x7fe0fc0cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe10170a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0fc15f310 0x7fe0fc0d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe10170a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc15f6f0 con 0x7fe0fc0d3980 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe10170a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc0b88f0 con 0x7fe0fc0d3520 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe10170a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe0fc0d2900 con 0x7fe0fc15f310 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0faffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0fc0d3520 0x7fe0fc162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0fb7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0fc15f310 0x7fe0fc0d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0faffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0fc0d3520 0x7fe0fc162160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38878/0 (socket says 172.21.15.192:38878) 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0faffd640 1 -- 172.21.15.192:0/1377261891 learned_addr learned my addr 172.21.15.192:0/1377261891 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0fa7fc640 1 --2- 172.21.15.192:0/1377261891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0fc0d3980 0x7fe0fc0cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0fb7fe640 1 -- 172.21.15.192:0/1377261891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0fc0d3520 msgr2=0x7fe0fc162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0fb7fe640 1 --2- 172.21.15.192:0/1377261891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0fc0d3520 0x7fe0fc162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0fb7fe640 1 -- 172.21.15.192:0/1377261891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0fc0d3980 msgr2=0x7fe0fc0cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0fb7fe640 1 --2- 172.21.15.192:0/1377261891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0fc0d3980 0x7fe0fc0cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:19.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0fb7fe640 1 -- 172.21.15.192:0/1377261891 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc0cddc0 con 0x7fe0fc15f310 2024-05-08T06:36:19.936 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0faffd640 1 --2- 172.21.15.192:0/1377261891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0fc0d3520 0x7fe0fc162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:19.936 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0fa7fc640 1 --2- 172.21.15.192:0/1377261891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0fc0d3980 0x7fe0fc0cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:19.936 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0fb7fe640 1 --2- 172.21.15.192:0/1377261891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0fc15f310 0x7fe0fc0d49d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400c780 tx=0x7fe0f400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:19.936 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0e37fe640 1 -- 172.21.15.192:0/1377261891 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4017070 con 0x7fe0fc15f310 2024-05-08T06:36:19.936 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0e37fe640 1 -- 172.21.15.192:0/1377261891 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0f400ce30 con 0x7fe0fc15f310 2024-05-08T06:36:19.936 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe10170a640 1 -- 172.21.15.192:0/1377261891 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0fc0ce0a0 con 0x7fe0fc15f310 2024-05-08T06:36:19.936 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.935+0000 7fe0e37fe640 1 -- 172.21.15.192:0/1377261891 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f400a860 con 0x7fe0fc15f310 2024-05-08T06:36:19.936 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.936+0000 7fe10170a640 1 -- 172.21.15.192:0/1377261891 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0fc14b570 con 0x7fe0fc15f310 2024-05-08T06:36:19.936 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.936+0000 7fe0e37fe640 1 -- 172.21.15.192:0/1377261891 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe0f401a070 con 0x7fe0fc15f310 2024-05-08T06:36:19.936 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.936+0000 7fe0e37fe640 1 -- 172.21.15.192:0/1377261891 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fe0f4007900 con 0x7fe0fc15f310 2024-05-08T06:36:19.936 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.937+0000 7fe10170a640 1 -- 172.21.15.192:0/1377261891 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0fc06a550 con 0x7fe0fc15f310 2024-05-08T06:36:19.937 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:19.939+0000 7fe0e37fe640 1 -- 172.21.15.192:0/1377261891 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe0f4021020 con 0x7fe0fc15f310 2024-05-08T06:36:20.009 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:20.011+0000 7fe10170a640 1 -- 172.21.15.192:0/1377261891 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b1615978-384e-436f-a343-60cc5bd9d058", "id": 7} v 0) -- 0x7fe0fc0b7790 con 0x7fe0fc15f310 2024-05-08T06:36:20.015 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:20.017+0000 7fe0e37fe640 1 -- 172.21.15.192:0/1377261891 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b1615978-384e-436f-a343-60cc5bd9d058", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fe0f400ac10 con 0x7fe0fc15f310 2024-05-08T06:36:20.016 INFO:teuthology.orchestra.run.smithi192.stdout:7 2024-05-08T06:36:20.017 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:20.018+0000 7fe10170a640 1 -- 172.21.15.192:0/1377261891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0fc15f310 msgr2=0x7fe0fc0d49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:20.017 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:20.018+0000 7fe10170a640 1 --2- 172.21.15.192:0/1377261891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0fc15f310 0x7fe0fc0d49d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400c780 tx=0x7fe0f400cc40 comp rx=0 tx=0).stop 2024-05-08T06:36:20.017 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:20.018+0000 7fe10170a640 1 -- 172.21.15.192:0/1377261891 shutdown_connections 2024-05-08T06:36:20.017 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:20.018+0000 7fe10170a640 1 --2- 172.21.15.192:0/1377261891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0fc15f310 0x7fe0fc0d49d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:20.017 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:20.018+0000 7fe10170a640 1 --2- 172.21.15.192:0/1377261891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0fc0d3980 0x7fe0fc0cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:20.017 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:20.018+0000 7fe10170a640 1 --2- 172.21.15.192:0/1377261891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0fc0d3520 0x7fe0fc162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:20.017 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:20.018+0000 7fe10170a640 1 -- 172.21.15.192:0/1377261891 >> 172.21.15.192:0/1377261891 conn(0x7fe0fc073e70 msgr2=0x7fe0fc0cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:20.017 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:20.018+0000 7fe10170a640 1 -- 172.21.15.192:0/1377261891 shutdown_connections 2024-05-08T06:36:20.017 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:36:20.018+0000 7fe10170a640 1 -- 172.21.15.192:0/1377261891 wait complete. 2024-05-08T06:36:20.027 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-08T06:36:20.027 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-08T06:36:20.030 INFO:tasks.ceph.osd.0:Started 2024-05-08T06:36:20.030 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T06:36:20.030 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T06:36:20.033 INFO:tasks.ceph.osd.1:Started 2024-05-08T06:36:20.033 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-08T06:36:20.033 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-08T06:36:20.035 INFO:tasks.ceph.osd.2:Started 2024-05-08T06:36:20.035 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-08T06:36:20.035 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-08T06:36:20.037 INFO:tasks.ceph.osd.3:Started 2024-05-08T06:36:20.038 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-08T06:36:20.038 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-08T06:36:20.040 INFO:tasks.ceph.osd.4:Started 2024-05-08T06:36:20.040 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T06:36:20.040 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T06:36:20.042 INFO:tasks.ceph.osd.5:Started 2024-05-08T06:36:20.042 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-08T06:36:20.042 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-08T06:36:20.044 INFO:tasks.ceph.osd.6:Started 2024-05-08T06:36:20.044 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-08T06:36:20.045 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-08T06:36:20.047 INFO:tasks.ceph.osd.7:Started 2024-05-08T06:36:20.047 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:36:20.070 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:20.071+0000 7f8df9f21040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T06:36:20.134 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:20.136+0000 7f5923bb8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T06:36:20.181 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.182+0000 7f90510cf640 1 Processor -- start 2024-05-08T06:36:20.181 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.182+0000 7f90510cf640 1 -- start start 2024-05-08T06:36:20.183 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f90510cf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f904c140070 0x7f904c071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:20.183 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f90510cf640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f904c142f20 0x7f904c1410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:20.183 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f90510cf640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f904c140450 con 0x7f904c0722f0 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f90510cf640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f904c072e50 con 0x7f904c142f20 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f90510cf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f904c072890 con 0x7f904c140070 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f904a59c640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f904c0722f0 0x7f904c142810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:32994/0 (socket says 172.21.15.163:32994) 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f904a59c640 1 -- 172.21.15.163:0/2288075862 learned_addr learned my addr 172.21.15.163:0/2288075862 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f904b59e640 1 --2- 172.21.15.163:0/2288075862 >> v2:172.21.15.192:3300/0 conn(0x7f904c142f20 0x7f904c1410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f904ad9d640 1 --2- 172.21.15.163:0/2288075862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f904c140070 0x7f904c071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f9049d9b640 1 -- 172.21.15.163:0/2288075862 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4146531975 0 0) 0x7f904c140450 con 0x7f904c0722f0 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f9049d9b640 1 -- 172.21.15.163:0/2288075862 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9038003520 con 0x7f904c0722f0 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f904b59e640 1 -- 172.21.15.163:0/2288075862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f904c140070 msgr2=0x7f904c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f904b59e640 1 --2- 172.21.15.163:0/2288075862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f904c140070 0x7f904c071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f904b59e640 1 -- 172.21.15.163:0/2288075862 >> v1:172.21.15.163:6789/0 conn(0x7f904c0722f0 legacy=0x7f904c142810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f904b59e640 1 -- 172.21.15.163:0/2288075862 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c072b70 con 0x7f904c142f20 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.183+0000 7f904ad9d640 1 --2- 172.21.15.163:0/2288075862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f904c140070 0x7f904c071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.184+0000 7f904b59e640 1 --2- 172.21.15.163:0/2288075862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f904c142f20 0x7f904c1410b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9044009d90 tx=0x7f904402a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62bbf03bb6439138 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:20.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.184+0000 7f9049d9b640 1 -- 172.21.15.163:0/2288075862 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044007740 con 0x7f904c142f20 2024-05-08T06:36:20.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.184+0000 7f9049d9b640 1 -- 172.21.15.163:0/2288075862 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90440078e0 con 0x7f904c142f20 2024-05-08T06:36:20.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.184+0000 7f9049d9b640 1 -- 172.21.15.163:0/2288075862 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044007cb0 con 0x7f904c142f20 2024-05-08T06:36:20.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.185+0000 7f90510cf640 1 -- 172.21.15.163:0/2288075862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f904c142f20 msgr2=0x7f904c1410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:20.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.185+0000 7f90510cf640 1 --2- 172.21.15.163:0/2288075862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f904c142f20 0x7f904c1410b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9044009d90 tx=0x7f904402a530 comp rx=0 tx=0).stop 2024-05-08T06:36:20.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.185+0000 7f90510cf640 1 -- 172.21.15.163:0/2288075862 shutdown_connections 2024-05-08T06:36:20.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.185+0000 7f90510cf640 1 --2- 172.21.15.163:0/2288075862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f904c142f20 0x7f904c1410b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:20.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.185+0000 7f90510cf640 1 --2- 172.21.15.163:0/2288075862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f904c140070 0x7f904c071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:20.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.185+0000 7f90510cf640 1 -- 172.21.15.163:0/2288075862 >> 172.21.15.163:0/2288075862 conn(0x7f904c0b4da0 msgr2=0x7f904c0b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:20.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.185+0000 7f90510cf640 1 -- 172.21.15.163:0/2288075862 shutdown_connections 2024-05-08T06:36:20.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.185+0000 7f90510cf640 1 -- 172.21.15.163:0/2288075862 wait complete. 2024-05-08T06:36:20.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.186+0000 7f90510cf640 1 Processor -- start 2024-05-08T06:36:20.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.186+0000 7f90510cf640 1 -- start start 2024-05-08T06:36:20.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.187+0000 7f90510cf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f904c0722f0 0x7f904c13cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:20.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.187+0000 7f90510cf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f904c140070 0x7f904c15f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:20.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.187+0000 7f90510cf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f904c142f20 0x7f904c0d2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:20.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.187+0000 7f90510cf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1415f0 con 0x7f904c142f20 2024-05-08T06:36:20.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.187+0000 7f90510cf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1435b0 con 0x7f904c0722f0 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.187+0000 7f90510cf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f904c141770 con 0x7f904c140070 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.187+0000 7f904b59e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f904c142f20 0x7f904c0d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.187+0000 7f904b59e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f904c142f20 0x7f904c0d2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43060/0 (socket says 172.21.15.163:43060) 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.187+0000 7f904b59e640 1 -- 172.21.15.163:0/863240631 learned_addr learned my addr 172.21.15.163:0/863240631 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.187+0000 7f904a59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f904c140070 0x7f904c15f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.187+0000 7f904ad9d640 1 --2- 172.21.15.163:0/863240631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f904c0722f0 0x7f904c13cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.188+0000 7f904b59e640 1 -- 172.21.15.163:0/863240631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f904c140070 msgr2=0x7f904c15f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.188+0000 7f904b59e640 1 --2- 172.21.15.163:0/863240631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f904c140070 0x7f904c15f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.188+0000 7f904b59e640 1 -- 172.21.15.163:0/863240631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f904c0722f0 msgr2=0x7f904c13cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.188+0000 7f904b59e640 1 --2- 172.21.15.163:0/863240631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f904c0722f0 0x7f904c13cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.188+0000 7f904b59e640 1 -- 172.21.15.163:0/863240631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c13d250 con 0x7f904c142f20 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.188+0000 7f904b59e640 1 --2- 172.21.15.163:0/863240631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f904c142f20 0x7f904c0d2140 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f904402ab20 tx=0x7f9044004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.188+0000 7f90437fe640 1 -- 172.21.15.163:0/863240631 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904403f070 con 0x7f904c142f20 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.188+0000 7f90437fe640 1 -- 172.21.15.163:0/863240631 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9044031bc0 con 0x7f904c142f20 2024-05-08T06:36:20.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.188+0000 7f90437fe640 1 -- 172.21.15.163:0/863240631 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044009450 con 0x7f904c142f20 2024-05-08T06:36:20.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.188+0000 7f90510cf640 1 -- 172.21.15.163:0/863240631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904c160300 con 0x7f904c142f20 2024-05-08T06:36:20.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.188+0000 7f90510cf640 1 -- 172.21.15.163:0/863240631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904c160020 con 0x7f904c142f20 2024-05-08T06:36:20.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.188+0000 7f90437fe640 1 -- 172.21.15.163:0/863240631 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9044044460 con 0x7f904c142f20 2024-05-08T06:36:20.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.188+0000 7f90437fe640 1 -- 172.21.15.163:0/863240631 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f904403b410 con 0x7f904c142f20 2024-05-08T06:36:20.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.189+0000 7f90510cf640 1 -- 172.21.15.163:0/863240631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904c0694f0 con 0x7f904c142f20 2024-05-08T06:36:20.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.191+0000 7f90437fe640 1 -- 172.21.15.163:0/863240631 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f904c0694f0 con 0x7f904c142f20 2024-05-08T06:36:20.234 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:20.235+0000 7f8df9f21040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T06:36:20.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.252+0000 7f90510cf640 1 -- 172.21.15.163:0/863240631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f904c156f40 con 0x7f904c142f20 2024-05-08T06:36:20.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.252+0000 7f90437fe640 1 -- 172.21.15.163:0/863240631 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f904c0694f0 con 0x7f904c142f20 2024-05-08T06:36:20.251 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:20.251 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":10,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:36:20.012645+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:36:20.252 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.253+0000 7f90510cf640 1 -- 172.21.15.163:0/863240631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f904c142f20 msgr2=0x7f904c0d2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:20.252 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.253+0000 7f90510cf640 1 --2- 172.21.15.163:0/863240631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f904c142f20 0x7f904c0d2140 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f904402ab20 tx=0x7f9044004a40 comp rx=0 tx=0).stop 2024-05-08T06:36:20.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.253+0000 7f90510cf640 1 -- 172.21.15.163:0/863240631 shutdown_connections 2024-05-08T06:36:20.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.253+0000 7f90510cf640 1 --2- 172.21.15.163:0/863240631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f904c142f20 0x7f904c0d2140 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:20.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.253+0000 7f90510cf640 1 --2- 172.21.15.163:0/863240631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f904c140070 0x7f904c15f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:20.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.253+0000 7f90510cf640 1 --2- 172.21.15.163:0/863240631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f904c0722f0 0x7f904c13cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:20.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.253+0000 7f90510cf640 1 -- 172.21.15.163:0/863240631 >> 172.21.15.163:0/863240631 conn(0x7f904c0b4da0 msgr2=0x7f904c0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:20.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.253+0000 7f90510cf640 1 -- 172.21.15.163:0/863240631 shutdown_connections 2024-05-08T06:36:20.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:20.253+0000 7f90510cf640 1 -- 172.21.15.163:0/863240631 wait complete. 2024-05-08T06:36:20.263 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-08T06:36:20.263 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-08T06:36:20.278 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:20.280+0000 7f5923bb8040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T06:36:20.498 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:20.499+0000 7f8df9f21040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T06:36:20.535 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:20.536+0000 7f5923bb8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T06:36:20.551 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:20.552+0000 7f8df9f21040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T06:36:20.587 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:20.589+0000 7f5923bb8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T06:36:20.632 INFO:tasks.ceph.osd.0.smithi163.stderr:2024-05-08T06:36:20.633+0000 7fca8f81e640 -1 Falling back to public interface 2024-05-08T06:36:20.638 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T06:36:20.640+0000 7f6b7b61e640 -1 Falling back to public interface 2024-05-08T06:36:20.650 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-05-08T06:36:20.652+0000 7f89c1e86640 -1 Falling back to public interface 2024-05-08T06:36:20.653 INFO:tasks.ceph.osd.2.smithi163.stderr:2024-05-08T06:36:20.654+0000 7fd9d48db640 -1 Falling back to public interface 2024-05-08T06:36:20.661 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-05-08T06:36:20.662+0000 7f5d3341c640 -1 Falling back to public interface 2024-05-08T06:36:20.662 INFO:tasks.ceph.osd.3.smithi163.stderr:2024-05-08T06:36:20.664+0000 7fa3cadbe640 -1 Falling back to public interface 2024-05-08T06:36:20.663 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T06:36:20.665+0000 7fb8d4c18640 -1 Falling back to public interface 2024-05-08T06:36:20.666 INFO:tasks.ceph.osd.1.smithi163.stderr:2024-05-08T06:36:20.667+0000 7f8db51ab640 -1 Falling back to public interface 2024-05-08T06:36:20.673 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:20.674+0000 7f8df9f21040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T06:36:20.707 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:20.709+0000 7f5923bb8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T06:36:20.970 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:20.971+0000 7f8df9f21040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T06:36:21.002 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:21.004+0000 7f5923bb8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T06:36:21.018 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:21.019+0000 7f8df9f21040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T06:36:21.050 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:21.051+0000 7f5923bb8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T06:36:21.069 DEBUG:teuthology.orchestra.run.smithi163:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T06:36:21.163 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.163+0000 7fcf74a25640 1 Processor -- start 2024-05-08T06:36:21.163 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.163+0000 7fcf74a25640 1 -- start start 2024-05-08T06:36:21.165 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.164+0000 7fcf74a25640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf70146cd0 0x7fcf700d28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:21.166 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.164+0000 7fcf74a25640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcf700b4ee0 0x7fcf700b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:21.166 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.164+0000 7fcf74a25640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf700d2e10 con 0x7fcf700cf9f0 2024-05-08T06:36:21.166 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.164+0000 7fcf74a25640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fcf700b5960 con 0x7fcf700b4ee0 2024-05-08T06:36:21.166 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.164+0000 7fcf74a25640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcf700d1b10 con 0x7fcf70146cd0 2024-05-08T06:36:21.166 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.165+0000 7fcf6e59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf70146cd0 0x7fcf700d28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:21.166 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.165+0000 7fcf6ed9d640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fcf700cf9f0 0x7fcf700b5ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:33590/0 (socket says 172.21.15.163:33590) 2024-05-08T06:36:21.166 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.165+0000 7fcf6ed9d640 1 -- 172.21.15.163:0/1791648195 learned_addr learned my addr 172.21.15.163:0/1791648195 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:21.166 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.165+0000 7fcf67fff640 1 --2- 172.21.15.163:0/1791648195 >> v2:172.21.15.192:3300/0 conn(0x7fcf700b4ee0 0x7fcf700b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:21.166 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.165+0000 7fcf6dd9b640 1 -- 172.21.15.163:0/1791648195 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1434631705 0 0) 0x7fcf700d2e10 con 0x7fcf700cf9f0 2024-05-08T06:36:21.166 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.165+0000 7fcf6dd9b640 1 -- 172.21.15.163:0/1791648195 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf5c003520 con 0x7fcf700cf9f0 2024-05-08T06:36:21.166 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.165+0000 7fcf6e59c640 1 -- 172.21.15.163:0/1791648195 >> v2:172.21.15.192:3300/0 conn(0x7fcf700b4ee0 msgr2=0x7fcf700b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:21.166 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.165+0000 7fcf6e59c640 1 --2- 172.21.15.163:0/1791648195 >> v2:172.21.15.192:3300/0 conn(0x7fcf700b4ee0 0x7fcf700b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:21.166 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.165+0000 7fcf6e59c640 1 -- 172.21.15.163:0/1791648195 >> v1:172.21.15.192:6789/0 conn(0x7fcf700cf9f0 legacy=0x7fcf700b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:21.167 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.165+0000 7fcf6e59c640 1 -- 172.21.15.163:0/1791648195 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf700d1df0 con 0x7fcf70146cd0 2024-05-08T06:36:21.167 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.165+0000 7fcf67fff640 1 --2- 172.21.15.163:0/1791648195 >> v2:172.21.15.192:3300/0 conn(0x7fcf700b4ee0 0x7fcf700b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:21.167 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.166+0000 7fcf6e59c640 1 --2- 172.21.15.163:0/1791648195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf70146cd0 0x7fcf700d28d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcf600097a0 tx=0x7fcf6002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=952f47c25165aa4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:21.167 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.166+0000 7fcf6dd9b640 1 -- 172.21.15.163:0/1791648195 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60037070 con 0x7fcf70146cd0 2024-05-08T06:36:21.167 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.166+0000 7fcf6dd9b640 1 -- 172.21.15.163:0/1791648195 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf60032440 con 0x7fcf70146cd0 2024-05-08T06:36:21.167 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.166+0000 7fcf6dd9b640 1 -- 172.21.15.163:0/1791648195 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60032810 con 0x7fcf70146cd0 2024-05-08T06:36:21.167 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.166+0000 7fcf74a25640 1 -- 172.21.15.163:0/1791648195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf70146cd0 msgr2=0x7fcf700d28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:21.167 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.166+0000 7fcf74a25640 1 --2- 172.21.15.163:0/1791648195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf70146cd0 0x7fcf700d28d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcf600097a0 tx=0x7fcf6002a180 comp rx=0 tx=0).stop 2024-05-08T06:36:21.167 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.166+0000 7fcf74a25640 1 -- 172.21.15.163:0/1791648195 shutdown_connections 2024-05-08T06:36:21.167 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.166+0000 7fcf74a25640 1 --2- 172.21.15.163:0/1791648195 >> v2:172.21.15.192:3300/0 conn(0x7fcf700b4ee0 0x7fcf700b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:21.167 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.166+0000 7fcf74a25640 1 --2- 172.21.15.163:0/1791648195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf70146cd0 0x7fcf700d28d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:21.167 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.166+0000 7fcf74a25640 1 -- 172.21.15.163:0/1791648195 >> 172.21.15.163:0/1791648195 conn(0x7fcf70068320 msgr2=0x7fcf70068730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:21.167 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.166+0000 7fcf74a25640 1 -- 172.21.15.163:0/1791648195 shutdown_connections 2024-05-08T06:36:21.167 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.166+0000 7fcf74a25640 1 -- 172.21.15.163:0/1791648195 wait complete. 2024-05-08T06:36:21.170 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.168+0000 7fcf74a25640 1 Processor -- start 2024-05-08T06:36:21.170 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.168+0000 7fcf74a25640 1 -- start start 2024-05-08T06:36:21.170 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.168+0000 7fcf74a25640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf700b4ee0 0x7fcf700b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:21.170 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.168+0000 7fcf74a25640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf700cf9f0 0x7fcf7015bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:21.170 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.168+0000 7fcf74a25640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf70146cd0 0x7fcf70138280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:21.170 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.168+0000 7fcf74a25640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcf70161a40 con 0x7fcf700cf9f0 2024-05-08T06:36:21.170 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.168+0000 7fcf74a25640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcf700d23b0 con 0x7fcf70146cd0 2024-05-08T06:36:21.170 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.168+0000 7fcf74a25640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcf70161bc0 con 0x7fcf700b4ee0 2024-05-08T06:36:21.170 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf67fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf700cf9f0 0x7fcf7015bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:21.170 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf67fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf700cf9f0 0x7fcf7015bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43072/0 (socket says 172.21.15.163:43072) 2024-05-08T06:36:21.170 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf67fff640 1 -- 172.21.15.163:0/2382617619 learned_addr learned my addr 172.21.15.163:0/2382617619 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:21.170 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf6e59c640 1 --2- 172.21.15.163:0/2382617619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf700b4ee0 0x7fcf700b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:21.170 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf6ed9d640 1 --2- 172.21.15.163:0/2382617619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf70146cd0 0x7fcf70138280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:21.170 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf67fff640 1 -- 172.21.15.163:0/2382617619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf700b4ee0 msgr2=0x7fcf700b6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:21.171 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf67fff640 1 --2- 172.21.15.163:0/2382617619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf700b4ee0 0x7fcf700b6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:21.171 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf67fff640 1 -- 172.21.15.163:0/2382617619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf70146cd0 msgr2=0x7fcf70138280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:21.171 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf67fff640 1 --2- 172.21.15.163:0/2382617619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf70146cd0 0x7fcf70138280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:21.171 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf67fff640 1 -- 172.21.15.163:0/2382617619 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7015c750 con 0x7fcf700cf9f0 2024-05-08T06:36:21.171 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf6e59c640 1 --2- 172.21.15.163:0/2382617619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf700b4ee0 0x7fcf700b6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:21.171 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf67fff640 1 --2- 172.21.15.163:0/2382617619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf700cf9f0 0x7fcf7015bde0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800e960 tx=0x7fcf5800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:21.171 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf66ffd640 1 -- 172.21.15.163:0/2382617619 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58017070 con 0x7fcf700cf9f0 2024-05-08T06:36:21.171 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf66ffd640 1 -- 172.21.15.163:0/2382617619 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf5800ca00 con 0x7fcf700cf9f0 2024-05-08T06:36:21.171 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf74a25640 1 -- 172.21.15.163:0/2382617619 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf701f8ca0 con 0x7fcf700cf9f0 2024-05-08T06:36:21.171 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf66ffd640 1 -- 172.21.15.163:0/2382617619 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800cdd0 con 0x7fcf700cf9f0 2024-05-08T06:36:21.171 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf74a25640 1 -- 172.21.15.163:0/2382617619 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf701f9260 con 0x7fcf700cf9f0 2024-05-08T06:36:21.171 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf66ffd640 1 -- 172.21.15.163:0/2382617619 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcf5801a070 con 0x7fcf700cf9f0 2024-05-08T06:36:21.171 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.169+0000 7fcf66ffd640 1 -- 172.21.15.163:0/2382617619 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fcf5801cb20 con 0x7fcf700cf9f0 2024-05-08T06:36:21.171 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.170+0000 7fcf74a25640 1 -- 172.21.15.163:0/2382617619 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf700b3cc0 con 0x7fcf700cf9f0 2024-05-08T06:36:21.173 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.174+0000 7fcf66ffd640 1 -- 172.21.15.163:0/2382617619 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fcf700b3cc0 con 0x7fcf700cf9f0 2024-05-08T06:36:21.182 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:21.183+0000 7f8df9f21040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T06:36:21.208 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:21.210+0000 7f5923bb8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T06:36:21.235 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.236+0000 7fcf74a25640 1 -- 172.21.15.163:0/2382617619 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf70050740 con 0x7fcf700cf9f0 2024-05-08T06:36:21.236 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.237+0000 7fcf66ffd640 1 -- 172.21.15.163:0/2382617619 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fcf5801c450 con 0x7fcf700cf9f0 2024-05-08T06:36:21.236 INFO:teuthology.misc.health.smithi163.stdout: 2024-05-08T06:36:21.236 INFO:teuthology.misc.health.smithi163.stdout:{"epoch":10,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:36:20.012645+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:36:21.237 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.238+0000 7fcf74a25640 1 -- 172.21.15.163:0/2382617619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf700cf9f0 msgr2=0x7fcf7015bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:21.238 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.238+0000 7fcf74a25640 1 --2- 172.21.15.163:0/2382617619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf700cf9f0 0x7fcf7015bde0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800e960 tx=0x7fcf5800ee20 comp rx=0 tx=0).stop 2024-05-08T06:36:21.238 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.238+0000 7fcf74a25640 1 -- 172.21.15.163:0/2382617619 shutdown_connections 2024-05-08T06:36:21.238 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.238+0000 7fcf74a25640 1 --2- 172.21.15.163:0/2382617619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf70146cd0 0x7fcf70138280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:21.238 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.238+0000 7fcf74a25640 1 --2- 172.21.15.163:0/2382617619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf700cf9f0 0x7fcf7015bde0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:21.238 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.238+0000 7fcf74a25640 1 --2- 172.21.15.163:0/2382617619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf700b4ee0 0x7fcf700b6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:21.238 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.238+0000 7fcf74a25640 1 -- 172.21.15.163:0/2382617619 >> 172.21.15.163:0/2382617619 conn(0x7fcf70068320 msgr2=0x7fcf700b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:21.238 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.238+0000 7fcf74a25640 1 -- 172.21.15.163:0/2382617619 shutdown_connections 2024-05-08T06:36:21.238 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:21.238+0000 7fcf74a25640 1 -- 172.21.15.163:0/2382617619 wait complete. 2024-05-08T06:36:21.246 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-08T06:36:21.324 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:21.325+0000 7f8df9f21040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T06:36:21.345 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:21.347+0000 7f5923bb8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T06:36:21.373 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:21.374+0000 7f8df9f21040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T06:36:21.394 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:21.396+0000 7f5923bb8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T06:36:21.443 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:21.444+0000 7f8df9f21040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T06:36:21.463 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:21.464+0000 7f5923bb8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T06:36:21.490 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:21.491+0000 7f8df9f21040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T06:36:21.509 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:21.511+0000 7f5923bb8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T06:36:21.969 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:21.969+0000 7f8df9f21040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T06:36:21.981 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:21.983+0000 7f5923bb8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T06:36:22.084 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:22.085+0000 7f8df9f21040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T06:36:22.101 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-08T06:36:22.102+0000 7f5923bb8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T06:36:23.099 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:23.100+0000 7f8d999c0640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-08T06:36:23.099 INFO:tasks.ceph.mgr.y.smithi163.stderr:2024-05-08T06:36:23.100+0000 7f8d999c0640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T06:36:24.097 INFO:tasks.ceph.osd.0.smithi163.stderr:2024-05-08T06:36:24.097+0000 7fca8f81e640 -1 osd.0 0 log_to_monitors true 2024-05-08T06:36:24.119 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T06:36:24.121+0000 7f6b7b61e640 -1 osd.5 0 log_to_monitors true 2024-05-08T06:36:24.160 INFO:tasks.ceph.osd.2.smithi163.stderr:2024-05-08T06:36:24.160+0000 7fd9d48db640 -1 osd.2 0 log_to_monitors true 2024-05-08T06:36:24.166 INFO:tasks.ceph.osd.3.smithi163.stderr:2024-05-08T06:36:24.167+0000 7fa3cadbe640 -1 osd.3 0 log_to_monitors true 2024-05-08T06:36:24.180 INFO:tasks.ceph.osd.1.smithi163.stderr:2024-05-08T06:36:24.180+0000 7f8db51ab640 -1 osd.1 0 log_to_monitors true 2024-05-08T06:36:24.199 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T06:36:24.201+0000 7fb8d4c18640 -1 osd.4 0 log_to_monitors true 2024-05-08T06:36:24.200 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-05-08T06:36:24.201+0000 7f89c1e86640 -1 osd.7 0 log_to_monitors true 2024-05-08T06:36:24.200 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-05-08T06:36:24.202+0000 7f5d3341c640 -1 osd.6 0 log_to_monitors true 2024-05-08T06:36:28.053 DEBUG:teuthology.orchestra.run.smithi163:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T06:36:28.093 INFO:tasks.ceph.osd.1.smithi163.stderr:2024-05-08T06:36:28.094+0000 7f8db1141640 -1 osd.1 0 waiting for initial osdmap 2024-05-08T06:36:28.106 INFO:tasks.ceph.osd.1.smithi163.stderr:2024-05-08T06:36:28.107+0000 7f8da8f61640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:36:28.149 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-05-08T06:36:28.150+0000 7f89bde1c640 -1 osd.7 0 waiting for initial osdmap 2024-05-08T06:36:28.164 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T06:36:28.165+0000 7f6b76f0c640 -1 osd.5 0 waiting for initial osdmap 2024-05-08T06:36:28.167 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-05-08T06:36:28.168+0000 7f89b5c3c640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:36:28.171 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T06:36:28.172+0000 7fb8d0580640 -1 osd.4 0 waiting for initial osdmap 2024-05-08T06:36:28.178 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-08T06:36:28.179+0000 7f6b6ed3d640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:36:28.185 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-08T06:36:28.186+0000 7fb8c83a0640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:36:28.185 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.186+0000 7f65c85b6640 1 Processor -- start 2024-05-08T06:36:28.186 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.187+0000 7f65c85b6640 1 -- start start 2024-05-08T06:36:28.188 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.188+0000 7f65c85b6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65c00b43e0 0x7f65c00b3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:28.188 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.188+0000 7f65c85b6640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f65c00cdad0 0x7f65c00cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:28.189 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.188+0000 7f65c85b6640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c00b3750 con 0x7f65c00c5fc0 2024-05-08T06:36:28.189 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.188+0000 7f65c85b6640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f65c00cd640 con 0x7f65c00cdad0 2024-05-08T06:36:28.189 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.188+0000 7f65c85b6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f65c00d0ef0 con 0x7f65c00b43e0 2024-05-08T06:36:28.189 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.188+0000 7f65c6b53640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f65c00cdad0 0x7f65c00cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:28.189 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.188+0000 7f65c6352640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f65c00c5fc0 0x7f65c00c63a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:33592/0 (socket says 172.21.15.163:33592) 2024-05-08T06:36:28.189 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.188+0000 7f65c6352640 1 -- 172.21.15.163:0/4063938421 learned_addr learned my addr 172.21.15.163:0/4063938421 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:28.189 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.188+0000 7f65c5b51640 1 --2- 172.21.15.163:0/4063938421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65c00b43e0 0x7f65c00b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:28.189 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.188+0000 7f65c6b53640 1 -- 172.21.15.163:0/4063938421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65c00b43e0 msgr2=0x7f65c00b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:28.190 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.188+0000 7f65c6b53640 1 --2- 172.21.15.163:0/4063938421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65c00b43e0 0x7f65c00b3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:28.190 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.188+0000 7f65c6b53640 1 -- 172.21.15.163:0/4063938421 >> v1:172.21.15.192:6789/0 conn(0x7f65c00c5fc0 legacy=0x7f65c00c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:28.190 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.188+0000 7f65c6b53640 1 -- 172.21.15.163:0/4063938421 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c00d11d0 con 0x7f65c00cdad0 2024-05-08T06:36:28.190 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.189+0000 7f65c6b53640 1 --2- 172.21.15.163:0/4063938421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65c00cdad0 0x7f65c00cdf60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009d90 tx=0x7f65b802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c885661c39d33644 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:28.190 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.189+0000 7f65c5350640 1 -- 172.21.15.163:0/4063938421 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8007740 con 0x7f65c00cdad0 2024-05-08T06:36:28.190 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.189+0000 7f65c5350640 1 -- 172.21.15.163:0/4063938421 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b80078e0 con 0x7f65c00cdad0 2024-05-08T06:36:28.190 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.189+0000 7f65c5350640 1 -- 172.21.15.163:0/4063938421 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8007cb0 con 0x7f65c00cdad0 2024-05-08T06:36:28.192 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-05-08T06:36:28.189+0000 7f5d2ee59640 -1 osd.6 0 waiting for initial osdmap 2024-05-08T06:36:28.197 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.190+0000 7f65c85b6640 1 -- 172.21.15.163:0/4063938421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65c00cdad0 msgr2=0x7f65c00cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:28.197 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.190+0000 7f65c85b6640 1 --2- 172.21.15.163:0/4063938421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65c00cdad0 0x7f65c00cdf60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009d90 tx=0x7f65b802a6f0 comp rx=0 tx=0).stop 2024-05-08T06:36:28.197 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.190+0000 7f65c85b6640 1 -- 172.21.15.163:0/4063938421 shutdown_connections 2024-05-08T06:36:28.197 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.190+0000 7f65c85b6640 1 --2- 172.21.15.163:0/4063938421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65c00cdad0 0x7f65c00cdf60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:28.197 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.190+0000 7f65c85b6640 1 --2- 172.21.15.163:0/4063938421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65c00b43e0 0x7f65c00b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:28.197 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.190+0000 7f65c85b6640 1 -- 172.21.15.163:0/4063938421 >> 172.21.15.163:0/4063938421 conn(0x7f65c009ba50 msgr2=0x7f65c009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:28.197 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.190+0000 7f65c85b6640 1 -- 172.21.15.163:0/4063938421 shutdown_connections 2024-05-08T06:36:28.197 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.191+0000 7f65c85b6640 1 -- 172.21.15.163:0/4063938421 wait complete. 2024-05-08T06:36:28.197 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.192+0000 7f65c85b6640 1 Processor -- start 2024-05-08T06:36:28.198 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.192+0000 7f65c85b6640 1 -- start start 2024-05-08T06:36:28.198 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c85b6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65c00b43e0 0x7f65c015d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:28.198 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c85b6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65c00c5fc0 0x7f65c00caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:28.198 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c85b6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c00cdad0 0x7f65c00ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:28.198 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c85b6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f65c009c840 con 0x7f65c00cdad0 2024-05-08T06:36:28.198 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c85b6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f65c009c6c0 con 0x7f65c00c5fc0 2024-05-08T06:36:28.198 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c85b6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f65c00b23a0 con 0x7f65c00b43e0 2024-05-08T06:36:28.198 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c6b53640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c00cdad0 0x7f65c00ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:28.198 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c6b53640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c00cdad0 0x7f65c00ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43162/0 (socket says 172.21.15.163:43162) 2024-05-08T06:36:28.198 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c6b53640 1 -- 172.21.15.163:0/1912966479 learned_addr learned my addr 172.21.15.163:0/1912966479 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:28.198 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c6352640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65c00b43e0 0x7f65c015d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:28.198 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c5b51640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65c00c5fc0 0x7f65c00caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:28.199 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c6b53640 1 -- 172.21.15.163:0/1912966479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65c00b43e0 msgr2=0x7f65c015d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:28.199 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c6b53640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65c00b43e0 0x7f65c015d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:28.199 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c6b53640 1 -- 172.21.15.163:0/1912966479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65c00c5fc0 msgr2=0x7f65c00caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:28.199 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c6b53640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65c00c5fc0 0x7f65c00caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:28.199 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c6b53640 1 -- 172.21.15.163:0/1912966479 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c00cb2e0 con 0x7f65c00cdad0 2024-05-08T06:36:28.199 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c5b51640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65c00c5fc0 0x7f65c00caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:28.199 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.193+0000 7f65c6352640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65c00b43e0 0x7f65c015d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:28.199 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.194+0000 7f65c6b53640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c00cdad0 0x7f65c00ce120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009e80 tx=0x7f65b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:28.199 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.197+0000 7f65b6ffd640 1 -- 172.21.15.163:0/1912966479 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8040030 con 0x7f65c00cdad0 2024-05-08T06:36:28.199 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.197+0000 7f65b6ffd640 1 -- 172.21.15.163:0/1912966479 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b8031970 con 0x7f65c00cdad0 2024-05-08T06:36:28.199 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.197+0000 7f65b6ffd640 1 -- 172.21.15.163:0/1912966479 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8031d40 con 0x7f65c00cdad0 2024-05-08T06:36:28.199 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.197+0000 7f65c85b6640 1 -- 172.21.15.163:0/1912966479 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c00cb5c0 con 0x7f65c00cdad0 2024-05-08T06:36:28.200 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.197+0000 7f65c85b6640 1 -- 172.21.15.163:0/1912966479 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c00cbb80 con 0x7f65c00cdad0 2024-05-08T06:36:28.200 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-05-08T06:36:28.200+0000 7f5d26c79640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:36:28.200 INFO:tasks.ceph.osd.3.smithi163.stderr:2024-05-08T06:36:28.198+0000 7fa3c6d54640 -1 osd.3 0 waiting for initial osdmap 2024-05-08T06:36:28.201 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.200+0000 7f65c85b6640 1 -- 172.21.15.163:0/1912966479 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c0050740 con 0x7f65c00cdad0 2024-05-08T06:36:28.201 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.200+0000 7f65b6ffd640 1 -- 172.21.15.163:0/1912966479 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 4) ==== 98145+0+0 (secure 0 0 0) 0x7f65b8043030 con 0x7f65c00cdad0 2024-05-08T06:36:28.201 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.202+0000 7f65b6ffd640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f65a4076290 0x7f65a4078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:28.201 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.202+0000 7f65b6ffd640 1 -- 172.21.15.163:0/1912966479 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f65b80bc060 con 0x7f65c00cdad0 2024-05-08T06:36:28.201 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.202+0000 7f65c6352640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f65a4076290 0x7f65a4078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:28.202 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.203+0000 7f65c6352640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f65a4076290 0x7f65a4078750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f65bc000f60 tx=0x7f65bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:28.204 INFO:tasks.ceph.osd.2.smithi163.stderr:2024-05-08T06:36:28.205+0000 7fd9d0871640 -1 osd.2 0 waiting for initial osdmap 2024-05-08T06:36:28.206 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.207+0000 7f65b6ffd640 1 -- 172.21.15.163:0/1912966479 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f65b80809e0 con 0x7f65c00cdad0 2024-05-08T06:36:28.209 INFO:tasks.ceph.osd.3.smithi163.stderr:2024-05-08T06:36:28.210+0000 7fa3beb74640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:36:28.217 INFO:tasks.ceph.osd.2.smithi163.stderr:2024-05-08T06:36:28.218+0000 7fd9c8691640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:36:28.224 INFO:tasks.ceph.osd.0.smithi163.stderr:2024-05-08T06:36:28.224+0000 7fca8b207640 -1 osd.0 0 waiting for initial osdmap 2024-05-08T06:36:28.234 INFO:tasks.ceph.osd.0.smithi163.stderr:2024-05-08T06:36:28.235+0000 7fca83038640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:36:28.392 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.393+0000 7f65c85b6640 1 -- 172.21.15.163:0/1912966479 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65c00b2a00 con 0x7f65c00cdad0 2024-05-08T06:36:28.393 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.394+0000 7f65b6ffd640 1 -- 172.21.15.163:0/1912966479 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f65b8084ce0 con 0x7f65c00cdad0 2024-05-08T06:36:28.393 INFO:teuthology.misc.health.smithi163.stdout: 2024-05-08T06:36:28.393 INFO:teuthology.misc.health.smithi163.stdout:{"epoch":12,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:36:26.101043+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:36:28.395 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.396+0000 7f65c85b6640 1 -- 172.21.15.163:0/1912966479 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f65a4076290 msgr2=0x7f65a4078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:28.395 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.396+0000 7f65c85b6640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f65a4076290 0x7f65a4078750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f65bc000f60 tx=0x7f65bc009040 comp rx=0 tx=0).stop 2024-05-08T06:36:28.395 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.396+0000 7f65c85b6640 1 -- 172.21.15.163:0/1912966479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c00cdad0 msgr2=0x7f65c00ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:28.396 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.396+0000 7f65c85b6640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c00cdad0 0x7f65c00ce120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009e80 tx=0x7f65b8004a40 comp rx=0 tx=0).stop 2024-05-08T06:36:28.396 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.396+0000 7f65c85b6640 1 -- 172.21.15.163:0/1912966479 shutdown_connections 2024-05-08T06:36:28.396 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.396+0000 7f65c85b6640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f65a4076290 0x7f65a4078750 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:28.396 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.396+0000 7f65c85b6640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c00cdad0 0x7f65c00ce120 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:28.396 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.396+0000 7f65c85b6640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65c00c5fc0 0x7f65c00caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:28.396 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.396+0000 7f65c85b6640 1 --2- 172.21.15.163:0/1912966479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65c00b43e0 0x7f65c015d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:28.396 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.396+0000 7f65c85b6640 1 -- 172.21.15.163:0/1912966479 >> 172.21.15.163:0/1912966479 conn(0x7f65c009ba50 msgr2=0x7f65c00d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:28.396 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.396+0000 7f65c85b6640 1 -- 172.21.15.163:0/1912966479 shutdown_connections 2024-05-08T06:36:28.396 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:28.396+0000 7f65c85b6640 1 -- 172.21.15.163:0/1912966479 wait complete. 2024-05-08T06:36:28.405 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-08T06:36:35.214 DEBUG:teuthology.orchestra.run.smithi163:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T06:36:35.318 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.318+0000 7ff5c5cb0640 1 Processor -- start 2024-05-08T06:36:35.318 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.318+0000 7ff5c5cb0640 1 -- start start 2024-05-08T06:36:35.319 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.319+0000 7ff5c5cb0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c0162b90 0x7ff5c00d5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:35.319 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.319+0000 7ff5c5cb0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5c00d3c60 0x7ff5c015f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:35.319 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.319+0000 7ff5c5cb0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c015fe10 con 0x7ff5c00d3800 2024-05-08T06:36:35.320 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.319+0000 7ff5c5cb0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c0162f70 con 0x7ff5c00d3c60 2024-05-08T06:36:35.320 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.319+0000 7ff5c5cb0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c0160f70 con 0x7ff5c0162b90 2024-05-08T06:36:35.320 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.319+0000 7ff5bffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5c00d3c60 0x7ff5c015f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:35.320 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.319+0000 7ff5beffd640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff5c00d3800 0x7ff5c015ee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45032/0 (socket says 172.21.15.163:45032) 2024-05-08T06:36:35.320 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.319+0000 7ff5beffd640 1 -- 172.21.15.163:0/3832485012 learned_addr learned my addr 172.21.15.163:0/3832485012 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:35.320 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.319+0000 7ff5bf7fe640 1 --2- 172.21.15.163:0/3832485012 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c0162b90 0x7ff5c00d5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:35.320 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.319+0000 7ff5bffff640 1 -- 172.21.15.163:0/3832485012 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c0162b90 msgr2=0x7ff5c00d5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:35.320 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.319+0000 7ff5bffff640 1 --2- 172.21.15.163:0/3832485012 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c0162b90 0x7ff5c00d5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.320 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.319+0000 7ff5bffff640 1 -- 172.21.15.163:0/3832485012 >> v1:172.21.15.163:6789/0 conn(0x7ff5c00d3800 legacy=0x7ff5c015ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:35.320 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.319+0000 7ff5bffff640 1 -- 172.21.15.163:0/3832485012 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c01600f0 con 0x7ff5c00d3c60 2024-05-08T06:36:35.325 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.320+0000 7ff5bffff640 1 --2- 172.21.15.163:0/3832485012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c00d3c60 0x7ff5c015f580 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009d90 tx=0x7ff5b802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a5fd0b9e7b7169 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:35.325 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.320+0000 7ff5be7fc640 1 -- 172.21.15.163:0/3832485012 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8007740 con 0x7ff5c00d3c60 2024-05-08T06:36:35.325 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.320+0000 7ff5be7fc640 1 -- 172.21.15.163:0/3832485012 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b80078e0 con 0x7ff5c00d3c60 2024-05-08T06:36:35.326 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.320+0000 7ff5be7fc640 1 -- 172.21.15.163:0/3832485012 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8007cb0 con 0x7ff5c00d3c60 2024-05-08T06:36:35.326 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.321+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/3832485012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c00d3c60 msgr2=0x7ff5c015f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:35.326 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.321+0000 7ff5c5cb0640 1 --2- 172.21.15.163:0/3832485012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c00d3c60 0x7ff5c015f580 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009d90 tx=0x7ff5b802a660 comp rx=0 tx=0).stop 2024-05-08T06:36:35.326 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.321+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/3832485012 shutdown_connections 2024-05-08T06:36:35.326 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.321+0000 7ff5c5cb0640 1 --2- 172.21.15.163:0/3832485012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c00d3c60 0x7ff5c015f580 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.326 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.321+0000 7ff5c5cb0640 1 --2- 172.21.15.163:0/3832485012 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c0162b90 0x7ff5c00d5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.326 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.321+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/3832485012 >> 172.21.15.163:0/3832485012 conn(0x7ff5c009e560 msgr2=0x7ff5c00d5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:35.326 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.321+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/3832485012 shutdown_connections 2024-05-08T06:36:35.326 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.321+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/3832485012 wait complete. 2024-05-08T06:36:35.327 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.322+0000 7ff5c5cb0640 1 Processor -- start 2024-05-08T06:36:35.327 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.322+0000 7ff5c5cb0640 1 -- start start 2024-05-08T06:36:35.327 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.323+0000 7ff5c5cb0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c00d3800 0x7ff5c0150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:35.327 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.323+0000 7ff5c5cb0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c00d3c60 0x7ff5c00cead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:35.327 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.323+0000 7ff5c5cb0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c0162b90 0x7ff5c00b8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:35.327 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.323+0000 7ff5c5cb0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c0160990 con 0x7ff5c0162b90 2024-05-08T06:36:35.327 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.323+0000 7ff5c5cb0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c01606b0 con 0x7ff5c00d3c60 2024-05-08T06:36:35.327 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.323+0000 7ff5c5cb0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c00d40c0 con 0x7ff5c00d3800 2024-05-08T06:36:35.327 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.323+0000 7ff5bffff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c0162b90 0x7ff5c00b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:35.327 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.323+0000 7ff5bffff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c0162b90 0x7ff5c00b8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46864/0 (socket says 172.21.15.163:46864) 2024-05-08T06:36:35.327 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.323+0000 7ff5bffff640 1 -- 172.21.15.163:0/1597389747 learned_addr learned my addr 172.21.15.163:0/1597389747 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:35.327 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.323+0000 7ff5bf7fe640 1 --2- 172.21.15.163:0/1597389747 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c00d3800 0x7ff5c0150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:35.327 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.323+0000 7ff5beffd640 1 --2- 172.21.15.163:0/1597389747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c00d3c60 0x7ff5c00cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:35.328 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.324+0000 7ff5bffff640 1 -- 172.21.15.163:0/1597389747 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c00d3800 msgr2=0x7ff5c0150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:35.328 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.324+0000 7ff5bffff640 1 --2- 172.21.15.163:0/1597389747 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c00d3800 0x7ff5c0150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.328 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.324+0000 7ff5bffff640 1 -- 172.21.15.163:0/1597389747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c00d3c60 msgr2=0x7ff5c00cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:35.328 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.324+0000 7ff5bffff640 1 --2- 172.21.15.163:0/1597389747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c00d3c60 0x7ff5c00cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.328 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.324+0000 7ff5bffff640 1 -- 172.21.15.163:0/1597389747 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c0150d60 con 0x7ff5c0162b90 2024-05-08T06:36:35.328 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.324+0000 7ff5bffff640 1 --2- 172.21.15.163:0/1597389747 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c0162b90 0x7ff5c00b8500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff5b802ac50 tx=0x7ff5b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:35.328 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.324+0000 7ff5a7fff640 1 -- 172.21.15.163:0/1597389747 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8040070 con 0x7ff5c0162b90 2024-05-08T06:36:35.328 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.324+0000 7ff5a7fff640 1 -- 172.21.15.163:0/1597389747 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b8031bc0 con 0x7ff5c0162b90 2024-05-08T06:36:35.328 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.324+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/1597389747 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c00b9aa0 con 0x7ff5c0162b90 2024-05-08T06:36:35.328 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.324+0000 7ff5a7fff640 1 -- 172.21.15.163:0/1597389747 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8045460 con 0x7ff5c0162b90 2024-05-08T06:36:35.329 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.324+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/1597389747 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c00b9200 con 0x7ff5c0162b90 2024-05-08T06:36:35.329 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.325+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/1597389747 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c0050740 con 0x7ff5c0162b90 2024-05-08T06:36:35.329 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.325+0000 7ff5a7fff640 1 -- 172.21.15.163:0/1597389747 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff5b8039040 con 0x7ff5c0162b90 2024-05-08T06:36:35.329 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.326+0000 7ff5a7fff640 1 --2- 172.21.15.163:0/1597389747 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff5ac076300 0x7ff5ac0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:35.329 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.326+0000 7ff5bf7fe640 1 --2- 172.21.15.163:0/1597389747 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff5ac076300 0x7ff5ac0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:35.329 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.326+0000 7ff5a7fff640 1 -- 172.21.15.163:0/1597389747 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff5b80b8a20 con 0x7ff5c0162b90 2024-05-08T06:36:35.329 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.326+0000 7ff5bf7fe640 1 --2- 172.21.15.163:0/1597389747 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff5ac076300 0x7ff5ac0787c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0006fd0 tx=0x7ff5b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:35.330 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.331+0000 7ff5a7fff640 1 -- 172.21.15.163:0/1597389747 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff5c0150d60 con 0x7ff5c0162b90 2024-05-08T06:36:35.511 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.511+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/1597389747 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5c00b7e40 con 0x7ff5c0162b90 2024-05-08T06:36:35.511 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.512+0000 7ff5a7fff640 1 -- 172.21.15.163:0/1597389747 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12908 (secure 0 0 0) 0x7ff5b8082080 con 0x7ff5c0162b90 2024-05-08T06:36:35.512 INFO:teuthology.misc.health.smithi163.stdout: 2024-05-08T06:36:35.512 INFO:teuthology.misc.health.smithi163.stdout:{"epoch":16,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:36:32.350930+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:36:35.514 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.514+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/1597389747 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff5ac076300 msgr2=0x7ff5ac0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:35.514 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.514+0000 7ff5c5cb0640 1 --2- 172.21.15.163:0/1597389747 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff5ac076300 0x7ff5ac0787c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0006fd0 tx=0x7ff5b0008040 comp rx=0 tx=0).stop 2024-05-08T06:36:35.514 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.514+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/1597389747 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c0162b90 msgr2=0x7ff5c00b8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:35.514 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.514+0000 7ff5c5cb0640 1 --2- 172.21.15.163:0/1597389747 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c0162b90 0x7ff5c00b8500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff5b802ac50 tx=0x7ff5b8004a40 comp rx=0 tx=0).stop 2024-05-08T06:36:35.514 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.514+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/1597389747 shutdown_connections 2024-05-08T06:36:35.514 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.514+0000 7ff5c5cb0640 1 --2- 172.21.15.163:0/1597389747 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff5ac076300 0x7ff5ac0787c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.514 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.514+0000 7ff5c5cb0640 1 --2- 172.21.15.163:0/1597389747 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c0162b90 0x7ff5c00b8500 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.514 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.514+0000 7ff5c5cb0640 1 --2- 172.21.15.163:0/1597389747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c00d3c60 0x7ff5c00cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.514 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.514+0000 7ff5c5cb0640 1 --2- 172.21.15.163:0/1597389747 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c00d3800 0x7ff5c0150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.514 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.514+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/1597389747 >> 172.21.15.163:0/1597389747 conn(0x7ff5c009e560 msgr2=0x7ff5c006d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:35.514 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.514+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/1597389747 shutdown_connections 2024-05-08T06:36:35.514 INFO:teuthology.misc.health.smithi163.stderr:2024-05-08T06:36:35.514+0000 7ff5c5cb0640 1 -- 172.21.15.163:0/1597389747 wait complete. 2024-05-08T06:36:35.524 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-08T06:36:35.524 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-08T06:36:35.525 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T06:36:35.525 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T06:36:35.527 INFO:tasks.ceph.mds.a:Started 2024-05-08T06:36:35.527 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T06:36:35.527 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T06:36:35.529 INFO:tasks.ceph.mds.c:Started 2024-05-08T06:36:35.529 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T06:36:35.529 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T06:36:35.532 INFO:tasks.ceph.mds.b:Started 2024-05-08T06:36:35.532 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T06:36:35.532 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T06:36:35.534 INFO:tasks.ceph.mds.d:Started 2024-05-08T06:36:35.535 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-08T06:36:35.535 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:36:35.603 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2024-05-08T06:36:35.603 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2024-05-08T06:36:35.617 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-08T06:36:35.618 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-08T06:36:35.640 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.641+0000 7fdab5166640 1 Processor -- start 2024-05-08T06:36:35.640 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.641+0000 7fdab5166640 1 -- start start 2024-05-08T06:36:35.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.642+0000 7fdab5166640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdab015ad00 0x7fdab015e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:35.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.642+0000 7fdab5166640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdab00b0f10 0x7fdab00d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:35.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.642+0000 7fdab5166640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab006f560 con 0x7fdab00d1490 2024-05-08T06:36:35.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.642+0000 7fdab5166640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fdab015b0e0 con 0x7fdab015ad00 2024-05-08T06:36:35.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.642+0000 7fdab5166640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdab006efa0 con 0x7fdab00b0f10 2024-05-08T06:36:35.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.642+0000 7fdaaed9d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdab015ad00 0x7fdab015e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:35.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.642+0000 7fdaaf59e640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fdab00d1490 0x7fdab006c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45048/0 (socket says 172.21.15.163:45048) 2024-05-08T06:36:35.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.642+0000 7fdaaed9d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdab015ad00 0x7fdab015e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46894/0 (socket says 172.21.15.163:46894) 2024-05-08T06:36:35.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.642+0000 7fdaaf59e640 1 -- 172.21.15.163:0/2495489862 learned_addr learned my addr 172.21.15.163:0/2495489862 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:35.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.643+0000 7fdaae59c640 1 --2- 172.21.15.163:0/2495489862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdab00b0f10 0x7fdab00d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:35.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.643+0000 7fdaaed9d640 1 -- 172.21.15.163:0/2495489862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdab00b0f10 msgr2=0x7fdab00d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:35.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.643+0000 7fdaaed9d640 1 --2- 172.21.15.163:0/2495489862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdab00b0f10 0x7fdab00d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.643+0000 7fdaaed9d640 1 -- 172.21.15.163:0/2495489862 >> v1:172.21.15.163:6789/0 conn(0x7fdab00d1490 legacy=0x7fdab006c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:35.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.643+0000 7fdaadd9b640 1 -- 172.21.15.163:0/2495489862 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3521883964 0 0) 0x7fdab006f560 con 0x7fdab00d1490 2024-05-08T06:36:35.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.643+0000 7fdaaed9d640 1 -- 172.21.15.163:0/2495489862 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab006f280 con 0x7fdab015ad00 2024-05-08T06:36:35.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.643+0000 7fdaae59c640 1 --2- 172.21.15.163:0/2495489862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdab00b0f10 0x7fdab00d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:35.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.643+0000 7fdaaed9d640 1 --2- 172.21.15.163:0/2495489862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab015ad00 0x7fdab015e280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00097a0 tx=0x7fdaa002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6cb6b722ef2491 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:35.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.643+0000 7fdaadd9b640 1 -- 172.21.15.163:0/2495489862 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0037070 con 0x7fdab015ad00 2024-05-08T06:36:35.645 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.644+0000 7fdaadd9b640 1 -- 172.21.15.163:0/2495489862 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa0032440 con 0x7fdab015ad00 2024-05-08T06:36:35.645 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.644+0000 7fdaadd9b640 1 -- 172.21.15.163:0/2495489862 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0032810 con 0x7fdab015ad00 2024-05-08T06:36:35.645 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.644+0000 7fdab5166640 1 -- 172.21.15.163:0/2495489862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab015ad00 msgr2=0x7fdab015e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:35.645 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.644+0000 7fdab5166640 1 --2- 172.21.15.163:0/2495489862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab015ad00 0x7fdab015e280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00097a0 tx=0x7fdaa002a180 comp rx=0 tx=0).stop 2024-05-08T06:36:35.645 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.644+0000 7fdab5166640 1 -- 172.21.15.163:0/2495489862 shutdown_connections 2024-05-08T06:36:35.645 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.644+0000 7fdab5166640 1 --2- 172.21.15.163:0/2495489862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdab00b0f10 0x7fdab00d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.645 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.644+0000 7fdab5166640 1 --2- 172.21.15.163:0/2495489862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab015ad00 0x7fdab015e280 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.645 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.644+0000 7fdab5166640 1 -- 172.21.15.163:0/2495489862 >> 172.21.15.163:0/2495489862 conn(0x7fdab00cf310 msgr2=0x7fdab0072c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:35.645 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.644+0000 7fdab5166640 1 -- 172.21.15.163:0/2495489862 shutdown_connections 2024-05-08T06:36:35.645 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.644+0000 7fdab5166640 1 -- 172.21.15.163:0/2495489862 wait complete. 2024-05-08T06:36:35.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.646+0000 7fdab5166640 1 Processor -- start 2024-05-08T06:36:35.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.646+0000 7fdab5166640 1 -- start start 2024-05-08T06:36:35.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdab5166640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab00b0f10 0x7fdab0151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:35.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdab5166640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdab00d1490 0x7fdab0156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:35.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdab5166640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdab015ad00 0x7fdab0157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:35.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdab5166640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdab00b1370 con 0x7fdab00b0f10 2024-05-08T06:36:35.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdab5166640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdab006fc90 con 0x7fdab00d1490 2024-05-08T06:36:35.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdab5166640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdab006ca30 con 0x7fdab015ad00 2024-05-08T06:36:35.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaaed9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab00b0f10 0x7fdab0151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:35.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaaed9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab00b0f10 0x7fdab0151250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46900/0 (socket says 172.21.15.163:46900) 2024-05-08T06:36:35.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaaed9d640 1 -- 172.21.15.163:0/1406377995 learned_addr learned my addr 172.21.15.163:0/1406377995 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:35.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaae59c640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdab00d1490 0x7fdab0156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:35.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaaf59e640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdab015ad00 0x7fdab0157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:35.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaaed9d640 1 -- 172.21.15.163:0/1406377995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdab015ad00 msgr2=0x7fdab0157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:35.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaaed9d640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdab015ad00 0x7fdab0157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaaed9d640 1 -- 172.21.15.163:0/1406377995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdab00d1490 msgr2=0x7fdab0156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:35.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaaed9d640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdab00d1490 0x7fdab0156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaaed9d640 1 -- 172.21.15.163:0/1406377995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab01545b0 con 0x7fdab00b0f10 2024-05-08T06:36:35.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaae59c640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdab00d1490 0x7fdab0156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:35.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaaed9d640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab00b0f10 0x7fdab0151250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0035040 tx=0x7fdaa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:35.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaa77fe640 1 -- 172.21.15.163:0/1406377995 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0037040 con 0x7fdab00b0f10 2024-05-08T06:36:35.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaa77fe640 1 -- 172.21.15.163:0/1406377995 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa0030770 con 0x7fdab00b0f10 2024-05-08T06:36:35.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.647+0000 7fdaa77fe640 1 -- 172.21.15.163:0/1406377995 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0030b80 con 0x7fdab00b0f10 2024-05-08T06:36:35.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.648+0000 7fdab5166640 1 -- 172.21.15.163:0/1406377995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab0154890 con 0x7fdab00b0f10 2024-05-08T06:36:35.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.648+0000 7fdab5166640 1 -- 172.21.15.163:0/1406377995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab0154e50 con 0x7fdab00b0f10 2024-05-08T06:36:35.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.649+0000 7fdab5166640 1 -- 172.21.15.163:0/1406377995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab0068480 con 0x7fdab00b0f10 2024-05-08T06:36:35.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.649+0000 7fdaa77fe640 1 -- 172.21.15.163:0/1406377995 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdaa0044020 con 0x7fdab00b0f10 2024-05-08T06:36:35.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.650+0000 7fdaa77fe640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fda900762d0 0x7fda90078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:35.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.650+0000 7fdaa77fe640 1 -- 172.21.15.163:0/1406377995 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fdaa00b78a0 con 0x7fdab00b0f10 2024-05-08T06:36:35.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.650+0000 7fdaae59c640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fda900762d0 0x7fda90078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:35.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.650+0000 7fdaae59c640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fda900762d0 0x7fda90078790 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdab01ea700 tx=0x7fda98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:35.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.654+0000 7fdaa77fe640 1 -- 172.21.15.163:0/1406377995 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdab0068480 con 0x7fdab00b0f10 2024-05-08T06:36:35.872 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.873+0000 7fdab5166640 1 -- 172.21.15.163:0/1406377995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdab006a550 con 0x7fdab00b0f10 2024-05-08T06:36:35.872 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.873+0000 7fdaa77fe640 1 -- 172.21.15.163:0/1406377995 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7fdaa0032e20 con 0x7fdab00b0f10 2024-05-08T06:36:35.873 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:35.873 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2413407842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2413407842},{"type":"v1","addr":"172.21.15.163:6835","nonce":2413407842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-08T06:36:35.873 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 2 2024-05-08T06:36:35.874 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.875+0000 7fdab5166640 1 -- 172.21.15.163:0/1406377995 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fda900762d0 msgr2=0x7fda90078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:35.874 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.875+0000 7fdab5166640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fda900762d0 0x7fda90078790 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdab01ea700 tx=0x7fda98008040 comp rx=0 tx=0).stop 2024-05-08T06:36:35.874 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.875+0000 7fdab5166640 1 -- 172.21.15.163:0/1406377995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab00b0f10 msgr2=0x7fdab0151250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:35.875 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.875+0000 7fdab5166640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab00b0f10 0x7fdab0151250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0035040 tx=0x7fdaa00046c0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.875 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.875+0000 7fdab5166640 1 -- 172.21.15.163:0/1406377995 shutdown_connections 2024-05-08T06:36:35.875 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.875+0000 7fdab5166640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fda900762d0 0x7fda90078790 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.875 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.875+0000 7fdab5166640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdab015ad00 0x7fdab0157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.875 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.875+0000 7fdab5166640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdab00d1490 0x7fdab0156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.875 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.875+0000 7fdab5166640 1 --2- 172.21.15.163:0/1406377995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdab00b0f10 0x7fdab0151250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:35.875 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.875+0000 7fdab5166640 1 -- 172.21.15.163:0/1406377995 >> 172.21.15.163:0/1406377995 conn(0x7fdab00cf310 msgr2=0x7fdab015b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:35.875 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.875+0000 7fdab5166640 1 -- 172.21.15.163:0/1406377995 shutdown_connections 2024-05-08T06:36:35.875 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:35.875+0000 7fdab5166640 1 -- 172.21.15.163:0/1406377995 wait complete. 2024-05-08T06:36:37.886 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:36:37.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:37.999+0000 7f889f420640 1 Processor -- start 2024-05-08T06:36:37.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.000+0000 7f889f420640 1 -- start start 2024-05-08T06:36:38.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.000+0000 7f889f420640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f88980b0430 0x7f88980b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.000+0000 7f889f420640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88980741d0 con 0x7f8898070ea0 2024-05-08T06:36:38.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.001+0000 7f889f420640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88980b5610 con 0x7f88980b0810 2024-05-08T06:36:38.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.001+0000 7f889f420640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f88980b1900 con 0x7f88980b0430 2024-05-08T06:36:38.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.001+0000 7f889d1bc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f88980b0430 0x7f88980b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.001+0000 7f889d1bc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f88980b0430 0x7f88980b4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46914/0 (socket says 172.21.15.163:46914) 2024-05-08T06:36:38.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.001+0000 7f889d1bc640 1 -- 172.21.15.163:0/4013971086 learned_addr learned my addr 172.21.15.163:0/4013971086 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:38.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.001+0000 7f889d1bc640 1 -- 172.21.15.163:0/4013971086 >> v1:172.21.15.192:6789/0 conn(0x7f88980b0810 legacy=0x7f889815f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:38.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.001+0000 7f889d1bc640 1 -- 172.21.15.163:0/4013971086 >> v1:172.21.15.163:6789/0 conn(0x7f8898070ea0 legacy=0x7f88980b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.001+0000 7f8897fff640 1 -- 172.21.15.163:0/4013971086 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2796494384 0 0) 0x7f88980741d0 con 0x7f8898070ea0 2024-05-08T06:36:38.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.001+0000 7f889d1bc640 1 -- 172.21.15.163:0/4013971086 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88980b6fd0 con 0x7f88980b0430 2024-05-08T06:36:38.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.002+0000 7f889d1bc640 1 --2- 172.21.15.163:0/4013971086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88980b0430 0x7f88980b4fa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8888009e50 tx=0x7f888802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6b6b6979077c9b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:38.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.002+0000 7f8897fff640 1 -- 172.21.15.163:0/4013971086 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888036070 con 0x7f88980b0430 2024-05-08T06:36:38.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.002+0000 7f8897fff640 1 -- 172.21.15.163:0/4013971086 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8888031600 con 0x7f88980b0430 2024-05-08T06:36:38.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.002+0000 7f8897fff640 1 -- 172.21.15.163:0/4013971086 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88880319d0 con 0x7f88980b0430 2024-05-08T06:36:38.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.002+0000 7f889f420640 1 -- 172.21.15.163:0/4013971086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88980b0430 msgr2=0x7f88980b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.002+0000 7f889f420640 1 --2- 172.21.15.163:0/4013971086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88980b0430 0x7f88980b4fa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8888009e50 tx=0x7f888802a680 comp rx=0 tx=0).stop 2024-05-08T06:36:38.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.002+0000 7f889f420640 1 -- 172.21.15.163:0/4013971086 shutdown_connections 2024-05-08T06:36:38.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.002+0000 7f889f420640 1 --2- 172.21.15.163:0/4013971086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88980b0430 0x7f88980b4fa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.002+0000 7f889f420640 1 -- 172.21.15.163:0/4013971086 >> 172.21.15.163:0/4013971086 conn(0x7f889806d430 msgr2=0x7f889806d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:38.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.003+0000 7f889f420640 1 -- 172.21.15.163:0/4013971086 shutdown_connections 2024-05-08T06:36:38.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.003+0000 7f889f420640 1 -- 172.21.15.163:0/4013971086 wait complete. 2024-05-08T06:36:38.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.004+0000 7f889f420640 1 Processor -- start 2024-05-08T06:36:38.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.004+0000 7f889f420640 1 -- start start 2024-05-08T06:36:38.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889f420640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8898070ea0 0x7f889815a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889f420640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88980b0430 0x7f88980d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889f420640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88980b0810 0x7f88981537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889f420640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f88980b6db0 con 0x7f88980b0430 2024-05-08T06:36:38.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889f420640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f88980b6c30 con 0x7f8898070ea0 2024-05-08T06:36:38.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889f420640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f88980b4340 con 0x7f88980b0810 2024-05-08T06:36:38.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889c9bb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88980b0430 0x7f88980d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889c9bb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88980b0430 0x7f88980d12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46924/0 (socket says 172.21.15.163:46924) 2024-05-08T06:36:38.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889c9bb640 1 -- 172.21.15.163:0/3692433435 learned_addr learned my addr 172.21.15.163:0/3692433435 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:38.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889d9bd640 1 --2- 172.21.15.163:0/3692433435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88980b0810 0x7f88981537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889d1bc640 1 --2- 172.21.15.163:0/3692433435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8898070ea0 0x7f889815a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889c9bb640 1 -- 172.21.15.163:0/3692433435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88980b0810 msgr2=0x7f88981537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889c9bb640 1 --2- 172.21.15.163:0/3692433435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88980b0810 0x7f88981537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889c9bb640 1 -- 172.21.15.163:0/3692433435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8898070ea0 msgr2=0x7f889815a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889c9bb640 1 --2- 172.21.15.163:0/3692433435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8898070ea0 0x7f889815a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889c9bb640 1 -- 172.21.15.163:0/3692433435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889815adb0 con 0x7f88980b0430 2024-05-08T06:36:38.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889c9bb640 1 --2- 172.21.15.163:0/3692433435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88980b0430 0x7f88980d12d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f888c00b3f0 tx=0x7f888c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:38.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f8895ffb640 1 -- 172.21.15.163:0/3692433435 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c015020 con 0x7f88980b0430 2024-05-08T06:36:38.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f8895ffb640 1 -- 172.21.15.163:0/3692433435 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f888c011830 con 0x7f88980b0430 2024-05-08T06:36:38.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f889f420640 1 -- 172.21.15.163:0/3692433435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88981f8d70 con 0x7f88980b0430 2024-05-08T06:36:38.007 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.005+0000 7f8895ffb640 1 -- 172.21.15.163:0/3692433435 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c011c20 con 0x7f88980b0430 2024-05-08T06:36:38.007 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.006+0000 7f889f420640 1 -- 172.21.15.163:0/3692433435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88981f9010 con 0x7f88980b0430 2024-05-08T06:36:38.007 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.007+0000 7f889f420640 1 -- 172.21.15.163:0/3692433435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889806a550 con 0x7f88980b0430 2024-05-08T06:36:38.007 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.007+0000 7f8895ffb640 1 -- 172.21.15.163:0/3692433435 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f888c018020 con 0x7f88980b0430 2024-05-08T06:36:38.007 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.008+0000 7f8895ffb640 1 --2- 172.21.15.163:0/3692433435 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8884076300 0x7f88840787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.008 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.008+0000 7f8895ffb640 1 -- 172.21.15.163:0/3692433435 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f888c096c40 con 0x7f88980b0430 2024-05-08T06:36:38.008 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.008+0000 7f889d1bc640 1 --2- 172.21.15.163:0/3692433435 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8884076300 0x7f88840787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.008 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.008+0000 7f889d1bc640 1 --2- 172.21.15.163:0/3692433435 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8884076300 0x7f88840787c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8888033000 tx=0x7f8888009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:38.011 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.012+0000 7f8895ffb640 1 -- 172.21.15.163:0/3692433435 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f888c01f030 con 0x7f88980b0430 2024-05-08T06:36:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.230+0000 7f889f420640 1 -- 172.21.15.163:0/3692433435 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8898073d40 con 0x7f88980b0430 2024-05-08T06:36:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.230+0000 7f8895ffb640 1 -- 172.21.15.163:0/3692433435 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3873 (secure 0 0 0) 0x7f888c060330 con 0x7f88980b0430 2024-05-08T06:36:38.230 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:38.230 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2413407842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2413407842},{"type":"v1","addr":"172.21.15.163:6835","nonce":2413407842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2405203133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2405203133},{"type":"v1","addr":"172.21.15.163:6837","nonce":2405203133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3301087897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3301087897},{"type":"v1","addr":"172.21.15.192:6833","nonce":3301087897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/354782184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":354782184},{"type":"v1","addr":"172.21.15.192:6835","nonce":354782184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-08T06:36:38.230 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 3 2024-05-08T06:36:38.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.232+0000 7f889f420640 1 -- 172.21.15.163:0/3692433435 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8884076300 msgr2=0x7f88840787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.232 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.232+0000 7f889f420640 1 --2- 172.21.15.163:0/3692433435 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8884076300 0x7f88840787c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8888033000 tx=0x7f8888009000 comp rx=0 tx=0).stop 2024-05-08T06:36:38.232 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.232+0000 7f889f420640 1 -- 172.21.15.163:0/3692433435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88980b0430 msgr2=0x7f88980d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.232 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.232+0000 7f889f420640 1 --2- 172.21.15.163:0/3692433435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88980b0430 0x7f88980d12d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f888c00b3f0 tx=0x7f888c00b8b0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.232 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.232+0000 7f889f420640 1 -- 172.21.15.163:0/3692433435 shutdown_connections 2024-05-08T06:36:38.232 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.232+0000 7f889f420640 1 --2- 172.21.15.163:0/3692433435 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8884076300 0x7f88840787c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.232 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.232+0000 7f889f420640 1 --2- 172.21.15.163:0/3692433435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88980b0810 0x7f88981537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.232 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.232+0000 7f889f420640 1 --2- 172.21.15.163:0/3692433435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88980b0430 0x7f88980d12d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.232 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.232+0000 7f889f420640 1 --2- 172.21.15.163:0/3692433435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8898070ea0 0x7f889815a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.232 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.232+0000 7f889f420640 1 -- 172.21.15.163:0/3692433435 >> 172.21.15.163:0/3692433435 conn(0x7f889806d430 msgr2=0x7f88980b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:38.232 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.232+0000 7f889f420640 1 -- 172.21.15.163:0/3692433435 shutdown_connections 2024-05-08T06:36:38.232 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.232+0000 7f889f420640 1 -- 172.21.15.163:0/3692433435 wait complete. 2024-05-08T06:36:38.243 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T06:36:38.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.344+0000 7ffa4e179640 1 Processor -- start 2024-05-08T06:36:38.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.344+0000 7ffa4e179640 1 -- start start 2024-05-08T06:36:38.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.345+0000 7ffa4e179640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa4815f680 0x7ffa480cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.345+0000 7ffa4e179640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ffa480d38a0 0x7ffa480d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.345+0000 7ffa4e179640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa480b4a10 con 0x7ffa480d4e40 2024-05-08T06:36:38.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.345+0000 7ffa4e179640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ffa480d3da0 con 0x7ffa480d38a0 2024-05-08T06:36:38.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.345+0000 7ffa4e179640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffa480d4080 con 0x7ffa4815f680 2024-05-08T06:36:38.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.345+0000 7ffa477fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa4815f680 0x7ffa480cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.345+0000 7ffa46ffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ffa480d38a0 0x7ffa480d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.345+0000 7ffa477fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa4815f680 0x7ffa480cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:52346/0 (socket says 172.21.15.163:52346) 2024-05-08T06:36:38.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.345+0000 7ffa477fe640 1 -- 172.21.15.163:0/3494681737 learned_addr learned my addr 172.21.15.163:0/3494681737 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:38.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.345+0000 7ffa47fff640 1 --1- 172.21.15.163:0/3494681737 >> v1:172.21.15.192:6789/0 conn(0x7ffa480d4e40 0x7ffa480b6d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:42156/0 (socket says 172.21.15.163:42156) 2024-05-08T06:36:38.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.345+0000 7ffa477fe640 1 -- 172.21.15.163:0/3494681737 >> v2:172.21.15.192:3300/0 conn(0x7ffa480d38a0 msgr2=0x7ffa480d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:38.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.345+0000 7ffa477fe640 1 --2- 172.21.15.163:0/3494681737 >> v2:172.21.15.192:3300/0 conn(0x7ffa480d38a0 0x7ffa480d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.345+0000 7ffa477fe640 1 -- 172.21.15.163:0/3494681737 >> v1:172.21.15.192:6789/0 conn(0x7ffa480d4e40 legacy=0x7ffa480b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.345+0000 7ffa477fe640 1 -- 172.21.15.163:0/3494681737 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa480d4360 con 0x7ffa4815f680 2024-05-08T06:36:38.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.346+0000 7ffa477fe640 1 --2- 172.21.15.163:0/3494681737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa4815f680 0x7ffa480cfc80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009870 tx=0x7ffa3802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d6a504e47d8d2a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:38.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.346+0000 7ffa467fc640 1 -- 172.21.15.163:0/3494681737 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38037070 con 0x7ffa4815f680 2024-05-08T06:36:38.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.346+0000 7ffa467fc640 1 -- 172.21.15.163:0/3494681737 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa38032440 con 0x7ffa4815f680 2024-05-08T06:36:38.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.346+0000 7ffa467fc640 1 -- 172.21.15.163:0/3494681737 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38032810 con 0x7ffa4815f680 2024-05-08T06:36:38.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.347+0000 7ffa4e179640 1 -- 172.21.15.163:0/3494681737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa4815f680 msgr2=0x7ffa480cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.347+0000 7ffa4e179640 1 --2- 172.21.15.163:0/3494681737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa4815f680 0x7ffa480cfc80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009870 tx=0x7ffa3802a270 comp rx=0 tx=0).stop 2024-05-08T06:36:38.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.347+0000 7ffa4e179640 1 -- 172.21.15.163:0/3494681737 shutdown_connections 2024-05-08T06:36:38.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.347+0000 7ffa4e179640 1 --2- 172.21.15.163:0/3494681737 >> v2:172.21.15.192:3300/0 conn(0x7ffa480d38a0 0x7ffa480d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.347+0000 7ffa4e179640 1 --2- 172.21.15.163:0/3494681737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa4815f680 0x7ffa480cfc80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.347+0000 7ffa4e179640 1 -- 172.21.15.163:0/3494681737 >> 172.21.15.163:0/3494681737 conn(0x7ffa48074080 msgr2=0x7ffa4806d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:38.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.347+0000 7ffa4e179640 1 -- 172.21.15.163:0/3494681737 shutdown_connections 2024-05-08T06:36:38.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.347+0000 7ffa4e179640 1 -- 172.21.15.163:0/3494681737 wait complete. 2024-05-08T06:36:38.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.348+0000 7ffa4e179640 1 Processor -- start 2024-05-08T06:36:38.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.348+0000 7ffa4e179640 1 -- start start 2024-05-08T06:36:38.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa4e179640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa480d38a0 0x7ffa480b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa4e179640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa480d4e40 0x7ffa4815fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa4e179640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa4815f680 0x7ffa481543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa4e179640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4815fa60 con 0x7ffa480d4e40 2024-05-08T06:36:38.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa4e179640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffa480d4920 con 0x7ffa480d38a0 2024-05-08T06:36:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa4e179640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffa480d2250 con 0x7ffa4815f680 2024-05-08T06:36:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa46ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa480d4e40 0x7ffa4815fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa46ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa480d4e40 0x7ffa4815fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46940/0 (socket says 172.21.15.163:46940) 2024-05-08T06:36:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa46ffd640 1 -- 172.21.15.163:0/3790949055 learned_addr learned my addr 172.21.15.163:0/3790949055 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa477fe640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa480d38a0 0x7ffa480b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa47fff640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa4815f680 0x7ffa481543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa46ffd640 1 -- 172.21.15.163:0/3790949055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa4815f680 msgr2=0x7ffa481543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa46ffd640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa4815f680 0x7ffa481543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa46ffd640 1 -- 172.21.15.163:0/3790949055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa480d38a0 msgr2=0x7ffa480b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa46ffd640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa480d38a0 0x7ffa480b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa46ffd640 1 -- 172.21.15.163:0/3790949055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa48160350 con 0x7ffa480d4e40 2024-05-08T06:36:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.349+0000 7ffa477fe640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa480d38a0 0x7ffa480b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.350+0000 7ffa46ffd640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa480d4e40 0x7ffa4815fbe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00e9e0 tx=0x7ffa3c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.350+0000 7ffa2ffff640 1 -- 172.21.15.163:0/3790949055 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c017070 con 0x7ffa480d4e40 2024-05-08T06:36:38.351 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.350+0000 7ffa2ffff640 1 -- 172.21.15.163:0/3790949055 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3c00c9d0 con 0x7ffa480d4e40 2024-05-08T06:36:38.351 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.350+0000 7ffa4e179640 1 -- 172.21.15.163:0/3790949055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa48160630 con 0x7ffa480d4e40 2024-05-08T06:36:38.351 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.350+0000 7ffa2ffff640 1 -- 172.21.15.163:0/3790949055 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c00cdc0 con 0x7ffa480d4e40 2024-05-08T06:36:38.351 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.350+0000 7ffa4e179640 1 -- 172.21.15.163:0/3790949055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa48160bf0 con 0x7ffa480d4e40 2024-05-08T06:36:38.352 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.351+0000 7ffa4e179640 1 -- 172.21.15.163:0/3790949055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa4806a4b0 con 0x7ffa480d4e40 2024-05-08T06:36:38.352 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.351+0000 7ffa2ffff640 1 -- 172.21.15.163:0/3790949055 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffa3c01a070 con 0x7ffa480d4e40 2024-05-08T06:36:38.352 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.352+0000 7ffa2ffff640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffa340762d0 0x7ffa34078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.352 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.352+0000 7ffa477fe640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffa340762d0 0x7ffa34078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.352 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.352+0000 7ffa2ffff640 1 -- 172.21.15.163:0/3790949055 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ffa3c097660 con 0x7ffa480d4e40 2024-05-08T06:36:38.352 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.352+0000 7ffa477fe640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffa340762d0 0x7ffa34078790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffa38034000 tx=0x7ffa3802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:38.355 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.356+0000 7ffa2ffff640 1 -- 172.21.15.163:0/3790949055 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffa3c061310 con 0x7ffa480d4e40 2024-05-08T06:36:38.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.537+0000 7ffa4e179640 1 -- 172.21.15.163:0/3790949055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ffa480c8670 con 0x7ffa480d4e40 2024-05-08T06:36:38.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.538+0000 7ffa2ffff640 1 -- 172.21.15.163:0/3790949055 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7ffa3c060c10 con 0x7ffa480d4e40 2024-05-08T06:36:38.537 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:38.537 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2024-05-08T06:36:38.537 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:36:38.537 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2024-05-08T06:36:38.538 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2024-05-08T06:36:38.538 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:36:38.538 INFO:teuthology.orchestra.run.smithi163.stdout:] 2024-05-08T06:36:38.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.540+0000 7ffa4e179640 1 -- 172.21.15.163:0/3790949055 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffa340762d0 msgr2=0x7ffa34078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.540+0000 7ffa4e179640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffa340762d0 0x7ffa34078790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffa38034000 tx=0x7ffa3802e000 comp rx=0 tx=0).stop 2024-05-08T06:36:38.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.540+0000 7ffa4e179640 1 -- 172.21.15.163:0/3790949055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa480d4e40 msgr2=0x7ffa4815fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.540+0000 7ffa4e179640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa480d4e40 0x7ffa4815fbe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00e9e0 tx=0x7ffa3c00eea0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.540+0000 7ffa4e179640 1 -- 172.21.15.163:0/3790949055 shutdown_connections 2024-05-08T06:36:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.540+0000 7ffa4e179640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffa340762d0 0x7ffa34078790 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.540+0000 7ffa4e179640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa4815f680 0x7ffa481543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.540+0000 7ffa4e179640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa480d4e40 0x7ffa4815fbe0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.540+0000 7ffa4e179640 1 --2- 172.21.15.163:0/3790949055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa480d38a0 0x7ffa480b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.540+0000 7ffa4e179640 1 -- 172.21.15.163:0/3790949055 >> 172.21.15.163:0/3790949055 conn(0x7ffa48074080 msgr2=0x7ffa480caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.540+0000 7ffa4e179640 1 -- 172.21.15.163:0/3790949055 shutdown_connections 2024-05-08T06:36:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.540+0000 7ffa4e179640 1 -- 172.21.15.163:0/3790949055 wait complete. 2024-05-08T06:36:38.550 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T06:36:38.550 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T06:36:38.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.656+0000 7f1a2d4e8640 1 Processor -- start 2024-05-08T06:36:38.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.656+0000 7f1a2d4e8640 1 -- start start 2024-05-08T06:36:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.657+0000 7f1a2d4e8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a280d4810 0x7f1a28169ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.657+0000 7f1a2d4e8640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a280b9cf0 con 0x7f1a280d28a0 2024-05-08T06:36:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.657+0000 7f1a2d4e8640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a280b9fd0 con 0x7f1a2815f4a0 2024-05-08T06:36:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.657+0000 7f1a2d4e8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1a280d3370 con 0x7f1a280d4810 2024-05-08T06:36:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.657+0000 7f1a267fc640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1a280d28a0 0x7f1a280b8200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45062/0 (socket says 172.21.15.163:45062) 2024-05-08T06:36:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.657+0000 7f1a267fc640 1 -- 172.21.15.163:0/3350869313 learned_addr learned my addr 172.21.15.163:0/3350869313 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.657+0000 7f1a277fe640 1 --2- 172.21.15.163:0/3350869313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a280d4810 0x7f1a28169ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.657+0000 7f1a25ffb640 1 -- 172.21.15.163:0/3350869313 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3914459785 0 0) 0x7f1a280b9cf0 con 0x7f1a280d28a0 2024-05-08T06:36:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.657+0000 7f1a25ffb640 1 -- 172.21.15.163:0/3350869313 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a10003520 con 0x7f1a280d28a0 2024-05-08T06:36:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.657+0000 7f1a25ffb640 1 -- 172.21.15.163:0/3350869313 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 855335357 0 0) 0x7f1a280b9fd0 con 0x7f1a2815f4a0 2024-05-08T06:36:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.657+0000 7f1a25ffb640 1 -- 172.21.15.163:0/3350869313 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a280b9cf0 con 0x7f1a2815f4a0 2024-05-08T06:36:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.658+0000 7f1a25ffb640 1 -- 172.21.15.163:0/3350869313 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1408209804 0 0) 0x7f1a10003520 con 0x7f1a280d28a0 2024-05-08T06:36:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.658+0000 7f1a277fe640 1 -- 172.21.15.163:0/3350869313 >> v1:172.21.15.192:6789/0 conn(0x7f1a2815f4a0 legacy=0x7f1a280d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.658+0000 7f1a277fe640 1 -- 172.21.15.163:0/3350869313 >> v1:172.21.15.163:6789/0 conn(0x7f1a280d28a0 legacy=0x7f1a280b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.658+0000 7f1a277fe640 1 -- 172.21.15.163:0/3350869313 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a280ba2b0 con 0x7f1a280d4810 2024-05-08T06:36:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.658+0000 7f1a277fe640 1 --2- 172.21.15.163:0/3350869313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a280d4810 0x7f1a28169ec0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009dc0 tx=0x7f1a2002a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b754b738d841eab8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.658+0000 7f1a25ffb640 1 -- 172.21.15.163:0/3350869313 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c0016f0 con 0x7f1a280d4810 2024-05-08T06:36:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.658+0000 7f1a25ffb640 1 -- 172.21.15.163:0/3350869313 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2002ae50 con 0x7f1a280d4810 2024-05-08T06:36:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.658+0000 7f1a25ffb640 1 -- 172.21.15.163:0/3350869313 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20007740 con 0x7f1a280d4810 2024-05-08T06:36:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.659+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3350869313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a280d4810 msgr2=0x7f1a28169ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.659+0000 7f1a2d4e8640 1 --2- 172.21.15.163:0/3350869313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a280d4810 0x7f1a28169ec0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009dc0 tx=0x7f1a2002a6f0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.659+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3350869313 shutdown_connections 2024-05-08T06:36:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.659+0000 7f1a2d4e8640 1 --2- 172.21.15.163:0/3350869313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a280d4810 0x7f1a28169ec0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.659+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3350869313 >> 172.21.15.163:0/3350869313 conn(0x7f1a28073e70 msgr2=0x7f1a28074280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:38.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.659+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3350869313 shutdown_connections 2024-05-08T06:36:38.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.659+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3350869313 wait complete. 2024-05-08T06:36:38.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.660+0000 7f1a2d4e8640 1 Processor -- start 2024-05-08T06:36:38.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.660+0000 7f1a2d4e8640 1 -- start start 2024-05-08T06:36:38.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.661+0000 7f1a2d4e8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a280d28a0 0x7f1a281f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.661+0000 7f1a2d4e8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a280d4810 0x7f1a281e7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.661+0000 7f1a2d4e8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a2815f4a0 0x7f1a281e8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.661+0000 7f1a2d4e8640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1a280b9aa0 con 0x7f1a280d4810 2024-05-08T06:36:38.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.661+0000 7f1a2d4e8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1a280b9920 con 0x7f1a280d28a0 2024-05-08T06:36:38.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.661+0000 7f1a2d4e8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1a280baf00 con 0x7f1a2815f4a0 2024-05-08T06:36:38.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.661+0000 7f1a267fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a280d4810 0x7f1a281e7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.661+0000 7f1a267fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a280d4810 0x7f1a281e7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46956/0 (socket says 172.21.15.163:46956) 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.661+0000 7f1a267fc640 1 -- 172.21.15.163:0/3947014964 learned_addr learned my addr 172.21.15.163:0/3947014964 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.661+0000 7f1a277fe640 1 --2- 172.21.15.163:0/3947014964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a2815f4a0 0x7f1a281e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.661+0000 7f1a26ffd640 1 --2- 172.21.15.163:0/3947014964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a280d28a0 0x7f1a281f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.662+0000 7f1a267fc640 1 -- 172.21.15.163:0/3947014964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a2815f4a0 msgr2=0x7f1a281e8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.662+0000 7f1a267fc640 1 --2- 172.21.15.163:0/3947014964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a2815f4a0 0x7f1a281e8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.662+0000 7f1a267fc640 1 -- 172.21.15.163:0/3947014964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a280d28a0 msgr2=0x7f1a281f8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.662+0000 7f1a267fc640 1 --2- 172.21.15.163:0/3947014964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a280d28a0 0x7f1a281f8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.662+0000 7f1a267fc640 1 -- 172.21.15.163:0/3947014964 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a280ce930 con 0x7f1a280d4810 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.662+0000 7f1a267fc640 1 --2- 172.21.15.163:0/3947014964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a280d4810 0x7f1a281e7af0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00c5b0 tx=0x7f1a1c00ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.662+0000 7f1a0f7fe640 1 -- 172.21.15.163:0/3947014964 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c017070 con 0x7f1a280d4810 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.662+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3947014964 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a2814ab40 con 0x7f1a280d4810 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.662+0000 7f1a0f7fe640 1 -- 172.21.15.163:0/3947014964 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a1c005ce0 con 0x7f1a280d4810 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.662+0000 7f1a0f7fe640 1 -- 172.21.15.163:0/3947014964 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c0072b0 con 0x7f1a280d4810 2024-05-08T06:36:38.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.662+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3947014964 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a2814b100 con 0x7f1a280d4810 2024-05-08T06:36:38.665 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.663+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3947014964 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a2806a550 con 0x7f1a280d4810 2024-05-08T06:36:38.665 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.664+0000 7f1a0f7fe640 1 -- 172.21.15.163:0/3947014964 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1a1c01a070 con 0x7f1a280d4810 2024-05-08T06:36:38.665 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.664+0000 7f1a0f7fe640 1 --2- 172.21.15.163:0/3947014964 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1a140762d0 0x7f1a14078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:38.665 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.664+0000 7f1a0f7fe640 1 -- 172.21.15.163:0/3947014964 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f1a1c096680 con 0x7f1a280d4810 2024-05-08T06:36:38.665 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.664+0000 7f1a26ffd640 1 --2- 172.21.15.163:0/3947014964 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1a140762d0 0x7f1a14078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:38.665 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.664+0000 7f1a26ffd640 1 --2- 172.21.15.163:0/3947014964 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1a140762d0 0x7f1a14078790 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a18001120 tx=0x7f1a18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:38.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.668+0000 7f1a0f7fe640 1 -- 172.21.15.163:0/3947014964 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1a1c05fcf0 con 0x7f1a280d4810 2024-05-08T06:36:38.848 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:38.849+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3947014964 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1a280d0470 con 0x7f1a280d4810 2024-05-08T06:36:39.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.622+0000 7f1a0f7fe640 1 -- 172.21.15.163:0/3947014964 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f1a1c063ff0 con 0x7f1a280d4810 2024-05-08T06:36:39.622 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2024-05-08T06:36:39.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.626+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3947014964 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1a140762d0 msgr2=0x7f1a14078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:39.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.626+0000 7f1a2d4e8640 1 --2- 172.21.15.163:0/3947014964 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1a140762d0 0x7f1a14078790 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a18001120 tx=0x7f1a18009040 comp rx=0 tx=0).stop 2024-05-08T06:36:39.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.626+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3947014964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a280d4810 msgr2=0x7f1a281e7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:39.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.626+0000 7f1a2d4e8640 1 --2- 172.21.15.163:0/3947014964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a280d4810 0x7f1a281e7af0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00c5b0 tx=0x7f1a1c00ca70 comp rx=0 tx=0).stop 2024-05-08T06:36:39.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.626+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3947014964 shutdown_connections 2024-05-08T06:36:39.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.626+0000 7f1a2d4e8640 1 --2- 172.21.15.163:0/3947014964 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1a140762d0 0x7f1a14078790 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:39.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.626+0000 7f1a2d4e8640 1 --2- 172.21.15.163:0/3947014964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a2815f4a0 0x7f1a281e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:39.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.626+0000 7f1a2d4e8640 1 --2- 172.21.15.163:0/3947014964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a280d4810 0x7f1a281e7af0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:39.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.626+0000 7f1a2d4e8640 1 --2- 172.21.15.163:0/3947014964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a280d28a0 0x7f1a281f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:39.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.626+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3947014964 >> 172.21.15.163:0/3947014964 conn(0x7f1a28073e70 msgr2=0x7f1a280b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:39.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.626+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3947014964 shutdown_connections 2024-05-08T06:36:39.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.627+0000 7f1a2d4e8640 1 -- 172.21.15.163:0/3947014964 wait complete. 2024-05-08T06:36:39.641 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T06:36:39.751 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.752+0000 7f1b63e83640 1 Processor -- start 2024-05-08T06:36:39.751 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.752+0000 7f1b63e83640 1 -- start start 2024-05-08T06:36:39.755 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b63e83640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1b5c15f600 0x7f1b5c0d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:39.755 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b63e83640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b5c0d5a10 0x7f1b5c0d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:39.755 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b63e83640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1b5c0b1ea0 0x7f1b5c0b2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:39.755 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b63e83640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f1b5c0d2830 con 0x7f1b5c15f600 2024-05-08T06:36:39.755 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b63e83640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f1b5c0d3070 con 0x7f1b5c0b1ea0 2024-05-08T06:36:39.755 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b63e83640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c0d3350 con 0x7f1b5c0d5a10 2024-05-08T06:36:39.755 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b61c1f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1b5c15f600 0x7f1b5c0d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:39.755 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b61c1f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1b5c15f600 0x7f1b5c0d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35378/0 (socket says 172.21.15.163:35378) 2024-05-08T06:36:39.755 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b61c1f640 1 -- 172.21.15.163:0/3174158908 learned_addr learned my addr 172.21.15.163:0/3174158908 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:39.755 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b62420640 1 --2- 172.21.15.163:0/3174158908 >> v2:172.21.15.192:3300/0 conn(0x7f1b5c0b1ea0 0x7f1b5c0b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:39.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b6141e640 1 --2- 172.21.15.163:0/3174158908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b5c0d5a10 0x7f1b5c0d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:39.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b61c1f640 1 -- 172.21.15.163:0/3174158908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b5c0d5a10 msgr2=0x7f1b5c0d2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:39.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b61c1f640 1 --2- 172.21.15.163:0/3174158908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b5c0d5a10 0x7f1b5c0d2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:39.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b61c1f640 1 -- 172.21.15.163:0/3174158908 >> v2:172.21.15.192:3300/0 conn(0x7f1b5c0b1ea0 msgr2=0x7f1b5c0b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:39.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b61c1f640 1 --2- 172.21.15.163:0/3174158908 >> v2:172.21.15.192:3300/0 conn(0x7f1b5c0b1ea0 0x7f1b5c0b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:39.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.753+0000 7f1b61c1f640 1 -- 172.21.15.163:0/3174158908 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c0d3630 con 0x7f1b5c15f600 2024-05-08T06:36:39.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.754+0000 7f1b61c1f640 1 --2- 172.21.15.163:0/3174158908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b5c15f600 0x7f1b5c0d1b40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c08f4c0 tx=0x7f1b5802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5600d9a74de8ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:39.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.754+0000 7f1b60c1d640 1 -- 172.21.15.163:0/3174158908 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58037070 con 0x7f1b5c15f600 2024-05-08T06:36:39.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.754+0000 7f1b60c1d640 1 -- 172.21.15.163:0/3174158908 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b58032440 con 0x7f1b5c15f600 2024-05-08T06:36:39.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.754+0000 7f1b60c1d640 1 -- 172.21.15.163:0/3174158908 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58032810 con 0x7f1b5c15f600 2024-05-08T06:36:39.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.754+0000 7f1b63e83640 1 -- 172.21.15.163:0/3174158908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b5c15f600 msgr2=0x7f1b5c0d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:39.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.754+0000 7f1b63e83640 1 --2- 172.21.15.163:0/3174158908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b5c15f600 0x7f1b5c0d1b40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c08f4c0 tx=0x7f1b5802a140 comp rx=0 tx=0).stop 2024-05-08T06:36:39.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.754+0000 7f1b63e83640 1 -- 172.21.15.163:0/3174158908 shutdown_connections 2024-05-08T06:36:39.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.754+0000 7f1b63e83640 1 --2- 172.21.15.163:0/3174158908 >> v2:172.21.15.192:3300/0 conn(0x7f1b5c0b1ea0 0x7f1b5c0b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:39.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.754+0000 7f1b63e83640 1 --2- 172.21.15.163:0/3174158908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b5c0d5a10 0x7f1b5c0d2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:39.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.754+0000 7f1b63e83640 1 --2- 172.21.15.163:0/3174158908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b5c15f600 0x7f1b5c0d1b40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:39.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.754+0000 7f1b63e83640 1 -- 172.21.15.163:0/3174158908 >> 172.21.15.163:0/3174158908 conn(0x7f1b5c09e5b0 msgr2=0x7f1b5c0d5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:39.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.755+0000 7f1b63e83640 1 -- 172.21.15.163:0/3174158908 shutdown_connections 2024-05-08T06:36:39.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.755+0000 7f1b63e83640 1 -- 172.21.15.163:0/3174158908 wait complete. 2024-05-08T06:36:39.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.756+0000 7f1b63e83640 1 Processor -- start 2024-05-08T06:36:39.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.756+0000 7f1b63e83640 1 -- start start 2024-05-08T06:36:39.760 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.756+0000 7f1b63e83640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b5c0b1ea0 0x7f1b5c0c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:39.760 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.756+0000 7f1b63e83640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b5c0d5a10 0x7f1b5c0ba930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:39.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.756+0000 7f1b63e83640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b5c15f600 0x7f1b5c153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:39.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.756+0000 7f1b63e83640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0d4160 con 0x7f1b5c15f600 2024-05-08T06:36:39.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.756+0000 7f1b63e83640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0d3c40 con 0x7f1b5c0d5a10 2024-05-08T06:36:39.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.756+0000 7f1b63e83640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c15f9e0 con 0x7f1b5c0b1ea0 2024-05-08T06:36:39.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.756+0000 7f1b62420640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b5c15f600 0x7f1b5c153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:39.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b62420640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b5c15f600 0x7f1b5c153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35388/0 (socket says 172.21.15.163:35388) 2024-05-08T06:36:39.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b62420640 1 -- 172.21.15.163:0/3569006559 learned_addr learned my addr 172.21.15.163:0/3569006559 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:39.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b61c1f640 1 --2- 172.21.15.163:0/3569006559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b5c0b1ea0 0x7f1b5c0c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:39.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b6141e640 1 --2- 172.21.15.163:0/3569006559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b5c0d5a10 0x7f1b5c0ba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:39.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b62420640 1 -- 172.21.15.163:0/3569006559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b5c0b1ea0 msgr2=0x7f1b5c0c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:39.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b62420640 1 --2- 172.21.15.163:0/3569006559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b5c0b1ea0 0x7f1b5c0c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:39.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b62420640 1 -- 172.21.15.163:0/3569006559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b5c0d5a10 msgr2=0x7f1b5c0ba930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:39.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b62420640 1 --2- 172.21.15.163:0/3569006559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b5c0d5a10 0x7f1b5c0ba930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:39.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b62420640 1 -- 172.21.15.163:0/3569006559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c153ed0 con 0x7f1b5c15f600 2024-05-08T06:36:39.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b62420640 1 --2- 172.21.15.163:0/3569006559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b5c15f600 0x7f1b5c153990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1b54007e60 tx=0x7f1b5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:39.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b527fc640 1 -- 172.21.15.163:0/3569006559 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54014620 con 0x7f1b5c15f600 2024-05-08T06:36:39.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b527fc640 1 -- 172.21.15.163:0/3569006559 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b54019070 con 0x7f1b5c15f600 2024-05-08T06:36:39.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b63e83640 1 -- 172.21.15.163:0/3569006559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5c0c9540 con 0x7f1b5c15f600 2024-05-08T06:36:39.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b527fc640 1 -- 172.21.15.163:0/3569006559 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54014a20 con 0x7f1b5c15f600 2024-05-08T06:36:39.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.757+0000 7f1b63e83640 1 -- 172.21.15.163:0/3569006559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5c0c9b00 con 0x7f1b5c15f600 2024-05-08T06:36:39.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.758+0000 7f1b63e83640 1 -- 172.21.15.163:0/3569006559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b5c06a550 con 0x7f1b5c15f600 2024-05-08T06:36:39.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.759+0000 7f1b527fc640 1 -- 172.21.15.163:0/3569006559 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1b54014bc0 con 0x7f1b5c15f600 2024-05-08T06:36:39.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.759+0000 7f1b527fc640 1 --2- 172.21.15.163:0/3569006559 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1b400762d0 0x7f1b40078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:39.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.759+0000 7f1b61c1f640 1 --2- 172.21.15.163:0/3569006559 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1b400762d0 0x7f1b40078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:39.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.759+0000 7f1b527fc640 1 -- 172.21.15.163:0/3569006559 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f1b54097bd0 con 0x7f1b5c15f600 2024-05-08T06:36:39.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.760+0000 7f1b61c1f640 1 --2- 172.21.15.163:0/3569006559 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1b400762d0 0x7f1b40078790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b58034000 tx=0x7f1b5802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:39.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.764+0000 7f1b527fc640 1 -- 172.21.15.163:0/3569006559 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1b5c153ed0 con 0x7f1b5c15f600 2024-05-08T06:36:39.947 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:39.948+0000 7f1b63e83640 1 -- 172.21.15.163:0/3569006559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1b5c0c8820 con 0x7f1b5c15f600 2024-05-08T06:36:40.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.624+0000 7f1b527fc640 1 -- 172.21.15.163:0/3569006559 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f1b540617a0 con 0x7f1b5c15f600 2024-05-08T06:36:40.624 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2024-05-08T06:36:40.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.628+0000 7f1b63e83640 1 -- 172.21.15.163:0/3569006559 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1b400762d0 msgr2=0x7f1b40078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:40.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.628+0000 7f1b63e83640 1 --2- 172.21.15.163:0/3569006559 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1b400762d0 0x7f1b40078790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b58034000 tx=0x7f1b5802e000 comp rx=0 tx=0).stop 2024-05-08T06:36:40.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.628+0000 7f1b63e83640 1 -- 172.21.15.163:0/3569006559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b5c15f600 msgr2=0x7f1b5c153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:40.628 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.628+0000 7f1b63e83640 1 --2- 172.21.15.163:0/3569006559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b5c15f600 0x7f1b5c153990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1b54007e60 tx=0x7f1b5400e4b0 comp rx=0 tx=0).stop 2024-05-08T06:36:40.628 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.629+0000 7f1b63e83640 1 -- 172.21.15.163:0/3569006559 shutdown_connections 2024-05-08T06:36:40.628 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.629+0000 7f1b63e83640 1 --2- 172.21.15.163:0/3569006559 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1b400762d0 0x7f1b40078790 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:40.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.629+0000 7f1b63e83640 1 --2- 172.21.15.163:0/3569006559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b5c15f600 0x7f1b5c153990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:40.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.629+0000 7f1b63e83640 1 --2- 172.21.15.163:0/3569006559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b5c0d5a10 0x7f1b5c0ba930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:40.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.629+0000 7f1b63e83640 1 --2- 172.21.15.163:0/3569006559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b5c0b1ea0 0x7f1b5c0c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:40.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.629+0000 7f1b63e83640 1 -- 172.21.15.163:0/3569006559 >> 172.21.15.163:0/3569006559 conn(0x7f1b5c09e5b0 msgr2=0x7f1b5c0d4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:40.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.629+0000 7f1b63e83640 1 -- 172.21.15.163:0/3569006559 shutdown_connections 2024-05-08T06:36:40.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.629+0000 7f1b63e83640 1 -- 172.21.15.163:0/3569006559 wait complete. 2024-05-08T06:36:40.650 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T06:36:40.754 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.755+0000 7fb69a809640 1 Processor -- start 2024-05-08T06:36:40.754 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.755+0000 7fb69a809640 1 -- start start 2024-05-08T06:36:40.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb69a809640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb694148e20 0x7fb69406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:40.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb69a809640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb694162ea0 0x7fb6940b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:40.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb69a809640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb69406de40 con 0x7fb6940b6910 2024-05-08T06:36:40.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb69a809640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fb6940b6d80 con 0x7fb694162ea0 2024-05-08T06:36:40.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb69a809640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb69406e0e0 con 0x7fb694148e20 2024-05-08T06:36:40.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb6937fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb694162ea0 0x7fb6940b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:40.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb698da6640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb6940b6910 0x7fb694071e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:59810/0 (socket says 172.21.15.163:59810) 2024-05-08T06:36:40.757 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb6937fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb694162ea0 0x7fb6940b63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35390/0 (socket says 172.21.15.163:35390) 2024-05-08T06:36:40.758 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb698da6640 1 -- 172.21.15.163:0/2330088845 learned_addr learned my addr 172.21.15.163:0/2330088845 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:40.758 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb693fff640 1 --2- 172.21.15.163:0/2330088845 >> v2:172.21.15.192:3300/0 conn(0x7fb694148e20 0x7fb69406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:40.758 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb6937fe640 1 -- 172.21.15.163:0/2330088845 >> v2:172.21.15.192:3300/0 conn(0x7fb694148e20 msgr2=0x7fb69406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:40.758 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb6937fe640 1 --2- 172.21.15.163:0/2330088845 >> v2:172.21.15.192:3300/0 conn(0x7fb694148e20 0x7fb69406d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:40.758 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb6937fe640 1 -- 172.21.15.163:0/2330088845 >> v1:172.21.15.163:6789/0 conn(0x7fb6940b6910 legacy=0x7fb694071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:40.758 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb6937fe640 1 -- 172.21.15.163:0/2330088845 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69406e3c0 con 0x7fb694162ea0 2024-05-08T06:36:40.758 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.756+0000 7fb692ffd640 1 -- 172.21.15.163:0/2330088845 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 471969306 0 0) 0x7fb69406de40 con 0x7fb6940b6910 2024-05-08T06:36:40.758 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.757+0000 7fb6937fe640 1 --2- 172.21.15.163:0/2330088845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb694162ea0 0x7fb6940b63d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb694064080 tx=0x7fb68802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fb714983ab5e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:40.758 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.757+0000 7fb692ffd640 1 -- 172.21.15.163:0/2330088845 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688037070 con 0x7fb694162ea0 2024-05-08T06:36:40.758 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.757+0000 7fb692ffd640 1 -- 172.21.15.163:0/2330088845 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb688032440 con 0x7fb694162ea0 2024-05-08T06:36:40.758 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.757+0000 7fb692ffd640 1 -- 172.21.15.163:0/2330088845 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688032810 con 0x7fb694162ea0 2024-05-08T06:36:40.758 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.757+0000 7fb69a809640 1 -- 172.21.15.163:0/2330088845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb694162ea0 msgr2=0x7fb6940b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:40.758 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.757+0000 7fb69a809640 1 --2- 172.21.15.163:0/2330088845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb694162ea0 0x7fb6940b63d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb694064080 tx=0x7fb68802a1a0 comp rx=0 tx=0).stop 2024-05-08T06:36:40.759 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.757+0000 7fb69a809640 1 -- 172.21.15.163:0/2330088845 shutdown_connections 2024-05-08T06:36:40.759 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.757+0000 7fb69a809640 1 --2- 172.21.15.163:0/2330088845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb694162ea0 0x7fb6940b63d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:40.759 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.757+0000 7fb69a809640 1 --2- 172.21.15.163:0/2330088845 >> v2:172.21.15.192:3300/0 conn(0x7fb694148e20 0x7fb69406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:40.759 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.757+0000 7fb69a809640 1 -- 172.21.15.163:0/2330088845 >> 172.21.15.163:0/2330088845 conn(0x7fb6940706c0 msgr2=0x7fb694070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:40.759 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.757+0000 7fb69a809640 1 -- 172.21.15.163:0/2330088845 shutdown_connections 2024-05-08T06:36:40.759 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.757+0000 7fb69a809640 1 -- 172.21.15.163:0/2330088845 wait complete. 2024-05-08T06:36:40.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.759+0000 7fb69a809640 1 Processor -- start 2024-05-08T06:36:40.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.759+0000 7fb69a809640 1 -- start start 2024-05-08T06:36:40.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.759+0000 7fb69a809640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6940b6910 0x7fb69415ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:40.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.759+0000 7fb69a809640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb694148e20 0x7fb6940b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:40.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.759+0000 7fb69a809640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb694162ea0 0x7fb6941468f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:40.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.759+0000 7fb69a809640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb694160cc0 con 0x7fb694162ea0 2024-05-08T06:36:40.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.759+0000 7fb69a809640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb69406e980 con 0x7fb694148e20 2024-05-08T06:36:40.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.759+0000 7fb69a809640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb694160e40 con 0x7fb6940b6910 2024-05-08T06:36:40.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb698da6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb694162ea0 0x7fb6941468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:40.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb698da6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb694162ea0 0x7fb6941468f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35400/0 (socket says 172.21.15.163:35400) 2024-05-08T06:36:40.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb698da6640 1 -- 172.21.15.163:0/2179959104 learned_addr learned my addr 172.21.15.163:0/2179959104 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:40.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb6937fe640 1 --2- 172.21.15.163:0/2179959104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb694148e20 0x7fb6940b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb693fff640 1 --2- 172.21.15.163:0/2179959104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6940b6910 0x7fb69415ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb698da6640 1 -- 172.21.15.163:0/2179959104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6940b6910 msgr2=0x7fb69415ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb698da6640 1 --2- 172.21.15.163:0/2179959104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6940b6910 0x7fb69415ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb698da6640 1 -- 172.21.15.163:0/2179959104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb694148e20 msgr2=0x7fb6940b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb698da6640 1 --2- 172.21.15.163:0/2179959104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb694148e20 0x7fb6940b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb698da6640 1 -- 172.21.15.163:0/2179959104 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6941f8640 con 0x7fb694162ea0 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb698da6640 1 --2- 172.21.15.163:0/2179959104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb694162ea0 0x7fb6941468f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00c7a0 tx=0x7fb68c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb690ff9640 1 -- 172.21.15.163:0/2179959104 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c018070 con 0x7fb694162ea0 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb690ff9640 1 -- 172.21.15.163:0/2179959104 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb68c00a5e0 con 0x7fb694162ea0 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb69a809640 1 -- 172.21.15.163:0/2179959104 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6941f8920 con 0x7fb694162ea0 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb690ff9640 1 -- 172.21.15.163:0/2179959104 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c00aa50 con 0x7fb694162ea0 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.760+0000 7fb69a809640 1 -- 172.21.15.163:0/2179959104 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb69415f110 con 0x7fb694162ea0 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.761+0000 7fb69a809640 1 -- 172.21.15.163:0/2179959104 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6941f8af0 con 0x7fb694162ea0 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.762+0000 7fb690ff9640 1 -- 172.21.15.163:0/2179959104 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb68c01b070 con 0x7fb694162ea0 2024-05-08T06:36:40.763 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.762+0000 7fb690ff9640 1 --2- 172.21.15.163:0/2179959104 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb6800762d0 0x7fb680078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:40.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.762+0000 7fb693fff640 1 --2- 172.21.15.163:0/2179959104 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb6800762d0 0x7fb680078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:40.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.763+0000 7fb690ff9640 1 -- 172.21.15.163:0/2179959104 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fb68c096b20 con 0x7fb694162ea0 2024-05-08T06:36:40.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.763+0000 7fb693fff640 1 --2- 172.21.15.163:0/2179959104 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb6800762d0 0x7fb680078790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb68400a970 tx=0x7fb684008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:40.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.767+0000 7fb690ff9640 1 -- 172.21.15.163:0/2179959104 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb68c060570 con 0x7fb694162ea0 2024-05-08T06:36:40.983 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:40.984+0000 7fb69a809640 1 -- 172.21.15.163:0/2179959104 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb69406f270 con 0x7fb694162ea0 2024-05-08T06:36:41.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.633+0000 7fb690ff9640 1 -- 172.21.15.163:0/2179959104 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fb69406f270 con 0x7fb694162ea0 2024-05-08T06:36:41.632 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T06:36:41.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.636+0000 7fb69a809640 1 -- 172.21.15.163:0/2179959104 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb6800762d0 msgr2=0x7fb680078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:41.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.636+0000 7fb69a809640 1 --2- 172.21.15.163:0/2179959104 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb6800762d0 0x7fb680078790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb68400a970 tx=0x7fb684008040 comp rx=0 tx=0).stop 2024-05-08T06:36:41.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.636+0000 7fb69a809640 1 -- 172.21.15.163:0/2179959104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb694162ea0 msgr2=0x7fb6941468f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:41.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.636+0000 7fb69a809640 1 --2- 172.21.15.163:0/2179959104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb694162ea0 0x7fb6941468f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00c7a0 tx=0x7fb68c00cc60 comp rx=0 tx=0).stop 2024-05-08T06:36:41.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.639+0000 7fb69a809640 1 -- 172.21.15.163:0/2179959104 shutdown_connections 2024-05-08T06:36:41.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.639+0000 7fb69a809640 1 --2- 172.21.15.163:0/2179959104 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb6800762d0 0x7fb680078790 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.639+0000 7fb69a809640 1 --2- 172.21.15.163:0/2179959104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb694162ea0 0x7fb6941468f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.639+0000 7fb69a809640 1 --2- 172.21.15.163:0/2179959104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb694148e20 0x7fb6940b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.639+0000 7fb69a809640 1 --2- 172.21.15.163:0/2179959104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6940b6910 0x7fb69415ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.639+0000 7fb69a809640 1 -- 172.21.15.163:0/2179959104 >> 172.21.15.163:0/2179959104 conn(0x7fb6940706c0 msgr2=0x7fb694161f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:41.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.639+0000 7fb69a809640 1 -- 172.21.15.163:0/2179959104 shutdown_connections 2024-05-08T06:36:41.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.640+0000 7fb69a809640 1 -- 172.21.15.163:0/2179959104 wait complete. 2024-05-08T06:36:41.662 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:36:41.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.763+0000 7f8e079af640 1 Processor -- start 2024-05-08T06:36:41.762 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.763+0000 7f8e079af640 1 -- start start 2024-05-08T06:36:41.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.764+0000 7f8e079af640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0015f640 0x7f8e000d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:41.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.764+0000 7f8e079af640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8e000d4050 0x7f8e000b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:41.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.764+0000 7f8e079af640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e000d1240 con 0x7f8e000b3f50 2024-05-08T06:36:41.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.764+0000 7f8e079af640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8e000d3750 con 0x7f8e000d4050 2024-05-08T06:36:41.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.764+0000 7f8e079af640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8e000d2750 con 0x7f8e0015f640 2024-05-08T06:36:41.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.764+0000 7f8e04f4a640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8e000b3f50 0x7f8e000d3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:59818/0 (socket says 172.21.15.163:59818) 2024-05-08T06:36:41.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.764+0000 7f8e04f4a640 1 -- 172.21.15.163:0/1776055878 learned_addr learned my addr 172.21.15.163:0/1776055878 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:41.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.764+0000 7f8e05f4c640 1 --2- 172.21.15.163:0/1776055878 >> v2:172.21.15.192:3300/0 conn(0x7f8e000d4050 0x7f8e000b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:41.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.764+0000 7f8e0574b640 1 --2- 172.21.15.163:0/1776055878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0015f640 0x7f8e000d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:41.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.764+0000 7f8df7fff640 1 -- 172.21.15.163:0/1776055878 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 893711161 0 0) 0x7f8e000d1240 con 0x7f8e000b3f50 2024-05-08T06:36:41.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 -- 172.21.15.163:0/1776055878 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8dec003520 con 0x7f8e000b3f50 2024-05-08T06:36:41.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 -- 172.21.15.163:0/1776055878 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3247046043 0 0) 0x7f8dec003520 con 0x7f8e000b3f50 2024-05-08T06:36:41.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 -- 172.21.15.163:0/1776055878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0015f640 msgr2=0x7f8e000d3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:41.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 --2- 172.21.15.163:0/1776055878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0015f640 0x7f8e000d3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 -- 172.21.15.163:0/1776055878 >> v2:172.21.15.192:3300/0 conn(0x7f8e000d4050 msgr2=0x7f8e000b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:41.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 --2- 172.21.15.163:0/1776055878 >> v2:172.21.15.192:3300/0 conn(0x7f8e000d4050 0x7f8e000b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 -- 172.21.15.163:0/1776055878 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e000d5c10 con 0x7f8e000b3f50 2024-05-08T06:36:41.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 -- 172.21.15.163:0/1776055878 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f8df00016f0 con 0x7f8e000b3f50 2024-05-08T06:36:41.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 -- 172.21.15.163:0/1776055878 >> v1:172.21.15.163:6789/0 conn(0x7f8e000b3f50 legacy=0x7f8e000d3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:41.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 --2- 172.21.15.163:0/1776055878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0015f640 0x7f8dec003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:41.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 --2- 172.21.15.163:0/1776055878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dec003fd0 0x7f8dec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:41.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 --2- 172.21.15.163:0/1776055878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dec004d40 0x7f8dec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:41.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 -- 172.21.15.163:0/1776055878 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8e000d3750 con 0x7f8dec003fd0 2024-05-08T06:36:41.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 -- 172.21.15.163:0/1776055878 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8e000d2750 con 0x7f8dec004d40 2024-05-08T06:36:41.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8df7fff640 1 -- 172.21.15.163:0/1776055878 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8dec00ba10 con 0x7f8e0015f640 2024-05-08T06:36:41.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8e0574b640 1 --2- 172.21.15.163:0/1776055878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0015f640 0x7f8dec003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:41.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8e04f4a640 1 --2- 172.21.15.163:0/1776055878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dec003fd0 0x7f8dec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:41.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8e05f4c640 1 --2- 172.21.15.163:0/1776055878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dec004d40 0x7f8dec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:41.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8e04f4a640 1 -- 172.21.15.163:0/1776055878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0015f640 msgr2=0x7f8dec003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:41.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8e04f4a640 1 --2- 172.21.15.163:0/1776055878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0015f640 0x7f8dec003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8e04f4a640 1 -- 172.21.15.163:0/1776055878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dec004d40 msgr2=0x7f8dec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:41.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8e04f4a640 1 --2- 172.21.15.163:0/1776055878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dec004d40 0x7f8dec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8e04f4a640 1 -- 172.21.15.163:0/1776055878 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dec003520 con 0x7f8dec003fd0 2024-05-08T06:36:41.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.765+0000 7f8e0574b640 1 --2- 172.21.15.163:0/1776055878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0015f640 0x7f8dec003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:41.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.766+0000 7f8e04f4a640 1 --2- 172.21.15.163:0/1776055878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dec003fd0 0x7f8dec004800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8df000c240 tx=0x7f8dec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6778a30ca450397e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:41.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.766+0000 7f8df7fff640 1 -- 172.21.15.163:0/1776055878 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0007d30 con 0x7f8dec003fd0 2024-05-08T06:36:41.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.766+0000 7f8df7fff640 1 -- 172.21.15.163:0/1776055878 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df0005ad0 con 0x7f8dec003fd0 2024-05-08T06:36:41.769 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.766+0000 7f8df7fff640 1 -- 172.21.15.163:0/1776055878 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0007070 con 0x7f8dec003fd0 2024-05-08T06:36:41.769 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.767+0000 7f8e079af640 1 -- 172.21.15.163:0/1776055878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dec003fd0 msgr2=0x7f8dec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:41.769 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.767+0000 7f8e079af640 1 --2- 172.21.15.163:0/1776055878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dec003fd0 0x7f8dec004800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8df000c240 tx=0x7f8dec005830 comp rx=0 tx=0).stop 2024-05-08T06:36:41.769 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.767+0000 7f8e05f4c640 1 -- 172.21.15.163:0/1776055878 reap_dead start 2024-05-08T06:36:41.769 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.767+0000 7f8e079af640 1 -- 172.21.15.163:0/1776055878 shutdown_connections 2024-05-08T06:36:41.769 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.767+0000 7f8e079af640 1 -- 172.21.15.163:0/1776055878 >> 172.21.15.163:0/1776055878 conn(0x7f8e0009e5b0 msgr2=0x7f8e000d5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:41.769 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.767+0000 7f8e079af640 1 -- 172.21.15.163:0/1776055878 shutdown_connections 2024-05-08T06:36:41.769 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.767+0000 7f8e079af640 1 -- 172.21.15.163:0/1776055878 wait complete. 2024-05-08T06:36:41.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.768+0000 7f8e079af640 1 Processor -- start 2024-05-08T06:36:41.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.768+0000 7f8e079af640 1 -- start start 2024-05-08T06:36:41.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e079af640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dec003fd0 0x7f8e000b9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:41.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e079af640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e000bac00 0x7f8e0009ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:41.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e079af640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0009f460 0x7f8e000cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:41.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e079af640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0015fa20 con 0x7f8e000bac00 2024-05-08T06:36:41.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e079af640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8e000d61d0 con 0x7f8dec003fd0 2024-05-08T06:36:41.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e079af640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8e000d3210 con 0x7f8e0009f460 2024-05-08T06:36:41.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e04f4a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e000bac00 0x7f8e0009ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:41.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e04f4a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e000bac00 0x7f8e0009ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35412/0 (socket says 172.21.15.163:35412) 2024-05-08T06:36:41.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e04f4a640 1 -- 172.21.15.163:0/1172891532 learned_addr learned my addr 172.21.15.163:0/1172891532 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:41.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e0574b640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dec003fd0 0x7f8e000b9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:41.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e05f4c640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0009f460 0x7f8e000cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:41.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e04f4a640 1 -- 172.21.15.163:0/1172891532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0009f460 msgr2=0x7f8e000cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:41.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e04f4a640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0009f460 0x7f8e000cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e04f4a640 1 -- 172.21.15.163:0/1172891532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dec003fd0 msgr2=0x7f8e000b9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:41.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e04f4a640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dec003fd0 0x7f8e000b9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e04f4a640 1 -- 172.21.15.163:0/1172891532 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e000b9580 con 0x7f8e000bac00 2024-05-08T06:36:41.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e05f4c640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0009f460 0x7f8e000cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:41.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e04f4a640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e000bac00 0x7f8e0009ef20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8df002ada0 tx=0x7f8e000d5de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:41.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8df5ffb640 1 -- 172.21.15.163:0/1172891532 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0037040 con 0x7f8e000bac00 2024-05-08T06:36:41.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8df5ffb640 1 -- 172.21.15.163:0/1172891532 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df0041070 con 0x7f8e000bac00 2024-05-08T06:36:41.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8e079af640 1 -- 172.21.15.163:0/1172891532 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e000a1060 con 0x7f8e000bac00 2024-05-08T06:36:41.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.769+0000 7f8df5ffb640 1 -- 172.21.15.163:0/1172891532 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df00016b0 con 0x7f8e000bac00 2024-05-08T06:36:41.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.770+0000 7f8e079af640 1 -- 172.21.15.163:0/1172891532 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e000a1620 con 0x7f8e000bac00 2024-05-08T06:36:41.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.771+0000 7f8e079af640 1 -- 172.21.15.163:0/1172891532 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0006a550 con 0x7f8e000bac00 2024-05-08T06:36:41.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.771+0000 7f8df5ffb640 1 -- 172.21.15.163:0/1172891532 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8df0007070 con 0x7f8e000bac00 2024-05-08T06:36:41.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.771+0000 7f8df5ffb640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8de4076300 0x7f8de40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:41.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.772+0000 7f8e0574b640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8de4076300 0x7f8de40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:41.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.772+0000 7f8df5ffb640 1 -- 172.21.15.163:0/1172891532 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f8df00b6f60 con 0x7f8e000bac00 2024-05-08T06:36:41.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.772+0000 7f8e0574b640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8de4076300 0x7f8de40787c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009bc0 tx=0x7f8dfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:41.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.776+0000 7f8df5ffb640 1 -- 172.21.15.163:0/1172891532 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8df00808c0 con 0x7f8e000bac00 2024-05-08T06:36:41.815 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T06:36:41.818 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T06:36:41.962 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.962+0000 7f8e079af640 1 -- 172.21.15.163:0/1172891532 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e0006d6d0 con 0x7f8e000bac00 2024-05-08T06:36:41.963 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.963+0000 7f8df5ffb640 1 -- 172.21.15.163:0/1172891532 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16642 (secure 0 0 0) 0x7f8df0005aa0 con 0x7f8e000bac00 2024-05-08T06:36:41.964 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:41.964 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":19,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:36:40.986044+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:36:38.853461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:36:39.952926+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:36:41.965 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.965+0000 7f8e079af640 1 -- 172.21.15.163:0/1172891532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8de4076300 msgr2=0x7f8de40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:41.966 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.965+0000 7f8e079af640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8de4076300 0x7f8de40787c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009bc0 tx=0x7f8dfc008040 comp rx=0 tx=0).stop 2024-05-08T06:36:41.966 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.965+0000 7f8e079af640 1 -- 172.21.15.163:0/1172891532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e000bac00 msgr2=0x7f8e0009ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:41.966 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.965+0000 7f8e079af640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e000bac00 0x7f8e0009ef20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8df002ada0 tx=0x7f8e000d5de0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.966 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.965+0000 7f8e079af640 1 -- 172.21.15.163:0/1172891532 shutdown_connections 2024-05-08T06:36:41.966 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.965+0000 7f8e079af640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8de4076300 0x7f8de40787c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.966 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.965+0000 7f8e079af640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e0009f460 0x7f8e000cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.966 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.965+0000 7f8e079af640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e000bac00 0x7f8e0009ef20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.966 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.965+0000 7f8e079af640 1 --2- 172.21.15.163:0/1172891532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dec003fd0 0x7f8e000b9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:41.966 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.966+0000 7f8e079af640 1 -- 172.21.15.163:0/1172891532 >> 172.21.15.163:0/1172891532 conn(0x7f8e0009e5b0 msgr2=0x7f8e000d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:41.966 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.966+0000 7f8e079af640 1 -- 172.21.15.163:0/1172891532 shutdown_connections 2024-05-08T06:36:41.966 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:41.966+0000 7f8e079af640 1 -- 172.21.15.163:0/1172891532 wait complete. 2024-05-08T06:36:41.976 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:36:42.076 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.077+0000 7f5ed2847640 1 Processor -- start 2024-05-08T06:36:42.076 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.077+0000 7f5ed2847640 1 -- start start 2024-05-08T06:36:42.077 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.077+0000 7f5ed2847640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5ecc0ae180 0x7f5ecc0b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:42.078 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.077+0000 7f5ed2847640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ecc0b46e0 0x7f5ecc0afc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:42.078 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.077+0000 7f5ed2847640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ecc072f80 con 0x7f5ecc15c020 2024-05-08T06:36:42.078 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.077+0000 7f5ed2847640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5ecc0ae700 con 0x7f5ecc0ae180 2024-05-08T06:36:42.078 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.077+0000 7f5ed2847640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5ecc073290 con 0x7f5ecc0b46e0 2024-05-08T06:36:42.078 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.078+0000 7f5ed0de4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ecc0b46e0 0x7f5ecc0afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:42.078 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.078+0000 7f5ecb7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5ecc0ae180 0x7f5ecc0b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:42.078 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.078+0000 7f5ecb7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5ecc0ae180 0x7f5ecc0b41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:60948/0 (socket says 172.21.15.163:60948) 2024-05-08T06:36:42.078 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.078+0000 7f5ed0de4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ecc0b46e0 0x7f5ecc0afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:34084/0 (socket says 172.21.15.163:34084) 2024-05-08T06:36:42.078 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.078+0000 7f5ecb7fe640 1 -- 172.21.15.163:0/4018297642 learned_addr learned my addr 172.21.15.163:0/4018297642 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:42.078 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.078+0000 7f5ecbfff640 1 --1- 172.21.15.163:0/4018297642 >> v1:172.21.15.192:6789/0 conn(0x7f5ecc15c020 0x7f5ecc073720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:34014/0 (socket says 172.21.15.163:34014) 2024-05-08T06:36:42.078 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.078+0000 7f5ed0de4640 1 -- 172.21.15.163:0/4018297642 >> v2:172.21.15.192:3300/0 conn(0x7f5ecc0ae180 msgr2=0x7f5ecc0b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:42.078 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.078+0000 7f5ed0de4640 1 --2- 172.21.15.163:0/4018297642 >> v2:172.21.15.192:3300/0 conn(0x7f5ecc0ae180 0x7f5ecc0b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.078 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.078+0000 7f5ed0de4640 1 -- 172.21.15.163:0/4018297642 >> v1:172.21.15.192:6789/0 conn(0x7f5ecc15c020 legacy=0x7f5ecc073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:42.079 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.078+0000 7f5ed0de4640 1 -- 172.21.15.163:0/4018297642 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc074430 con 0x7f5ecc0b46e0 2024-05-08T06:36:42.079 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.078+0000 7f5ecaffd640 1 -- 172.21.15.163:0/4018297642 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3574358469 0 0) 0x7f5ebc001110 con 0x7f5ecc15c020 2024-05-08T06:36:42.079 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.078+0000 7f5ecb7fe640 1 --2- 172.21.15.163:0/4018297642 >> v2:172.21.15.192:3300/0 conn(0x7f5ecc0ae180 0x7f5ecc0b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:42.080 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.079+0000 7f5ed0de4640 1 --2- 172.21.15.163:0/4018297642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ecc0b46e0 0x7f5ecc0afc00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009d90 tx=0x7f5ec402a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1644897a8229c567 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:42.080 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.079+0000 7f5ecaffd640 1 -- 172.21.15.163:0/4018297642 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4007740 con 0x7f5ecc0b46e0 2024-05-08T06:36:42.080 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.079+0000 7f5ecaffd640 1 -- 172.21.15.163:0/4018297642 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ec40078e0 con 0x7f5ecc0b46e0 2024-05-08T06:36:42.080 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.079+0000 7f5ecaffd640 1 -- 172.21.15.163:0/4018297642 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4007cb0 con 0x7f5ecc0b46e0 2024-05-08T06:36:42.080 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.080+0000 7f5ed2847640 1 -- 172.21.15.163:0/4018297642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ecc0b46e0 msgr2=0x7f5ecc0afc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.080+0000 7f5ed2847640 1 --2- 172.21.15.163:0/4018297642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ecc0b46e0 0x7f5ecc0afc00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009d90 tx=0x7f5ec402a530 comp rx=0 tx=0).stop 2024-05-08T06:36:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.080+0000 7f5ed2847640 1 -- 172.21.15.163:0/4018297642 shutdown_connections 2024-05-08T06:36:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.080+0000 7f5ed2847640 1 --2- 172.21.15.163:0/4018297642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ecc0b46e0 0x7f5ecc0afc00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.080+0000 7f5ed2847640 1 --2- 172.21.15.163:0/4018297642 >> v2:172.21.15.192:3300/0 conn(0x7f5ecc0ae180 0x7f5ecc0b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.080+0000 7f5ed2847640 1 -- 172.21.15.163:0/4018297642 >> 172.21.15.163:0/4018297642 conn(0x7f5ecc06c500 msgr2=0x7f5ecc06c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.080+0000 7f5ed2847640 1 -- 172.21.15.163:0/4018297642 shutdown_connections 2024-05-08T06:36:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.080+0000 7f5ed2847640 1 -- 172.21.15.163:0/4018297642 wait complete. 2024-05-08T06:36:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.081+0000 7f5ed2847640 1 Processor -- start 2024-05-08T06:36:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.081+0000 7f5ed2847640 1 -- start start 2024-05-08T06:36:42.084 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ed2847640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ecc0ae180 0x7f5ecc147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:42.084 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ed2847640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ecc0b46e0 0x7f5ecc15ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:42.084 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ed2847640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ecc15c020 0x7f5ecc15b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:42.084 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ed2847640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc0d0860 con 0x7f5ecc0ae180 2024-05-08T06:36:42.084 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ed2847640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc15c400 con 0x7f5ecc15c020 2024-05-08T06:36:42.084 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ed2847640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5ecc0b4b70 con 0x7f5ecc0b46e0 2024-05-08T06:36:42.084 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ecbfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ecc0ae180 0x7f5ecc147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:42.084 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ecbfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ecc0ae180 0x7f5ecc147a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35420/0 (socket says 172.21.15.163:35420) 2024-05-08T06:36:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ecbfff640 1 -- 172.21.15.163:0/841144467 learned_addr learned my addr 172.21.15.163:0/841144467 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ed0de4640 1 --2- 172.21.15.163:0/841144467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ecc15c020 0x7f5ecc15b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ecb7fe640 1 --2- 172.21.15.163:0/841144467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ecc0b46e0 0x7f5ecc15ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ecbfff640 1 -- 172.21.15.163:0/841144467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ecc0b46e0 msgr2=0x7f5ecc15ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ecbfff640 1 --2- 172.21.15.163:0/841144467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ecc0b46e0 0x7f5ecc15ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ecbfff640 1 -- 172.21.15.163:0/841144467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ecc15c020 msgr2=0x7f5ecc15b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ecbfff640 1 --2- 172.21.15.163:0/841144467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ecc15c020 0x7f5ecc15b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ecbfff640 1 -- 172.21.15.163:0/841144467 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc147f40 con 0x7f5ecc0ae180 2024-05-08T06:36:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.082+0000 7f5ecbfff640 1 --2- 172.21.15.163:0/841144467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ecc0ae180 0x7f5ecc147a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00c3e0 tx=0x7f5ebc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.083+0000 7f5ec8ff9640 1 -- 172.21.15.163:0/841144467 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc0074e0 con 0x7f5ecc0ae180 2024-05-08T06:36:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.083+0000 7f5ec8ff9640 1 -- 172.21.15.163:0/841144467 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ebc0047e0 con 0x7f5ecc0ae180 2024-05-08T06:36:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.083+0000 7f5ed2847640 1 -- 172.21.15.163:0/841144467 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc1484a0 con 0x7f5ecc0ae180 2024-05-08T06:36:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.083+0000 7f5ec8ff9640 1 -- 172.21.15.163:0/841144467 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc004bd0 con 0x7f5ecc0ae180 2024-05-08T06:36:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.083+0000 7f5ed2847640 1 -- 172.21.15.163:0/841144467 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc148a60 con 0x7f5ecc0ae180 2024-05-08T06:36:42.086 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.084+0000 7f5ed2847640 1 -- 172.21.15.163:0/841144467 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ecc06a550 con 0x7f5ecc0ae180 2024-05-08T06:36:42.086 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.084+0000 7f5ec8ff9640 1 -- 172.21.15.163:0/841144467 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5ebc011070 con 0x7f5ecc0ae180 2024-05-08T06:36:42.086 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.085+0000 7f5ec8ff9640 1 --2- 172.21.15.163:0/841144467 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5eb80762d0 0x7f5eb8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:42.086 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.085+0000 7f5ecb7fe640 1 --2- 172.21.15.163:0/841144467 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5eb80762d0 0x7f5eb8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:42.086 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.085+0000 7f5ec8ff9640 1 -- 172.21.15.163:0/841144467 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f5ebc0970b0 con 0x7f5ecc0ae180 2024-05-08T06:36:42.086 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.085+0000 7f5ecb7fe640 1 --2- 172.21.15.163:0/841144467 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5eb80762d0 0x7f5eb8078790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc1f63f0 tx=0x7f5ec0009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:42.088 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.089+0000 7f5ec8ff9640 1 -- 172.21.15.163:0/841144467 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5ecc072f80 con 0x7f5ecc0ae180 2024-05-08T06:36:42.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.270+0000 7f5ed2847640 1 -- 172.21.15.163:0/841144467 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ecc160980 con 0x7f5ecc0ae180 2024-05-08T06:36:42.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.271+0000 7f5ec8ff9640 1 -- 172.21.15.163:0/841144467 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16642 (secure 0 0 0) 0x7f5ebc0603c0 con 0x7f5ecc0ae180 2024-05-08T06:36:42.270 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:42.271 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":19,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:36:40.986044+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:36:38.853461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:36:39.952926+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:36:42.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.273+0000 7f5ed2847640 1 -- 172.21.15.163:0/841144467 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5eb80762d0 msgr2=0x7f5eb8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:42.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.273+0000 7f5ed2847640 1 --2- 172.21.15.163:0/841144467 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5eb80762d0 0x7f5eb8078790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc1f63f0 tx=0x7f5ec0009240 comp rx=0 tx=0).stop 2024-05-08T06:36:42.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.273+0000 7f5ed2847640 1 -- 172.21.15.163:0/841144467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ecc0ae180 msgr2=0x7f5ecc147a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:42.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.273+0000 7f5ed2847640 1 --2- 172.21.15.163:0/841144467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ecc0ae180 0x7f5ecc147a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00c3e0 tx=0x7f5ebc00c8a0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.273+0000 7f5ed2847640 1 -- 172.21.15.163:0/841144467 shutdown_connections 2024-05-08T06:36:42.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.273+0000 7f5ed2847640 1 --2- 172.21.15.163:0/841144467 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5eb80762d0 0x7f5eb8078790 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.273+0000 7f5ed2847640 1 --2- 172.21.15.163:0/841144467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ecc15c020 0x7f5ecc15b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.273+0000 7f5ed2847640 1 --2- 172.21.15.163:0/841144467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ecc0b46e0 0x7f5ecc15ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.273+0000 7f5ed2847640 1 --2- 172.21.15.163:0/841144467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ecc0ae180 0x7f5ecc147a00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.273+0000 7f5ed2847640 1 -- 172.21.15.163:0/841144467 >> 172.21.15.163:0/841144467 conn(0x7f5ecc06c500 msgr2=0x7f5ecc15e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:42.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.273+0000 7f5ed2847640 1 -- 172.21.15.163:0/841144467 shutdown_connections 2024-05-08T06:36:42.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.273+0000 7f5ed2847640 1 -- 172.21.15.163:0/841144467 wait complete. 2024-05-08T06:36:42.284 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T06:36:42.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.387+0000 7fa47ca99640 1 Processor -- start 2024-05-08T06:36:42.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.387+0000 7fa47ca99640 1 -- start start 2024-05-08T06:36:42.389 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47ca99640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa47815f5d0 0x7fa4780d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:42.389 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47ca99640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4780d0fc0 con 0x7fa4780d0a00 2024-05-08T06:36:42.389 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47ca99640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4780d12a0 con 0x7fa4780d2100 2024-05-08T06:36:42.389 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47ca99640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa4780d3820 con 0x7fa47815f5d0 2024-05-08T06:36:42.389 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa475d9b640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa4780d0a00 0x7fa4780d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:59830/0 (socket says 172.21.15.163:59830) 2024-05-08T06:36:42.389 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa475d9b640 1 -- 172.21.15.163:0/1094438711 learned_addr learned my addr 172.21.15.163:0/1094438711 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:42.389 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47659c640 1 --2- 172.21.15.163:0/1094438711 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa47815f5d0 0x7fa4780d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:42.390 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 -- 172.21.15.163:0/1094438711 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3529519953 0 0) 0x7fa4780d0fc0 con 0x7fa4780d0a00 2024-05-08T06:36:42.390 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 -- 172.21.15.163:0/1094438711 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa460003520 con 0x7fa4780d0a00 2024-05-08T06:36:42.390 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 -- 172.21.15.163:0/1094438711 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 92722844 0 0) 0x7fa460003520 con 0x7fa4780d0a00 2024-05-08T06:36:42.390 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 -- 172.21.15.163:0/1094438711 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa47815f5d0 msgr2=0x7fa4780d5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:42.390 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 --2- 172.21.15.163:0/1094438711 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa47815f5d0 0x7fa4780d5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.390 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 -- 172.21.15.163:0/1094438711 >> v1:172.21.15.192:6789/0 conn(0x7fa4780d2100 legacy=0x7fa4780d3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:42.390 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 -- 172.21.15.163:0/1094438711 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4780d31d0 con 0x7fa4780d0a00 2024-05-08T06:36:42.390 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 -- 172.21.15.163:0/1094438711 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7fa46c0016f0 con 0x7fa4780d0a00 2024-05-08T06:36:42.390 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 -- 172.21.15.163:0/1094438711 >> v1:172.21.15.163:6789/0 conn(0x7fa4780d0a00 legacy=0x7fa4780d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:42.390 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 --2- 172.21.15.163:0/1094438711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4600039e0 0x7fa460003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:42.390 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 --2- 172.21.15.163:0/1094438711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa460004800 0x7fa460004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:42.390 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 --2- 172.21.15.163:0/1094438711 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa47815f5d0 0x7fa4600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:42.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 -- 172.21.15.163:0/1094438711 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa4780d3820 con 0x7fa460004800 2024-05-08T06:36:42.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 -- 172.21.15.163:0/1094438711 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa46000ba70 con 0x7fa4600039e0 2024-05-08T06:36:42.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47559a640 1 -- 172.21.15.163:0/1094438711 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa46000bc70 con 0x7fa47815f5d0 2024-05-08T06:36:42.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa475d9b640 1 --2- 172.21.15.163:0/1094438711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa460004800 0x7fa460004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:42.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.388+0000 7fa47659c640 1 --2- 172.21.15.163:0/1094438711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4600039e0 0x7fa460003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:42.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.389+0000 7fa476d9d640 1 --2- 172.21.15.163:0/1094438711 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa47815f5d0 0x7fa4600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:42.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.389+0000 7fa475d9b640 1 -- 172.21.15.163:0/1094438711 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa47815f5d0 msgr2=0x7fa4600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:42.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.389+0000 7fa475d9b640 1 --2- 172.21.15.163:0/1094438711 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa47815f5d0 0x7fa4600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.389+0000 7fa475d9b640 1 -- 172.21.15.163:0/1094438711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4600039e0 msgr2=0x7fa460003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:42.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.389+0000 7fa475d9b640 1 --2- 172.21.15.163:0/1094438711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4600039e0 0x7fa460003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.389+0000 7fa475d9b640 1 -- 172.21.15.163:0/1094438711 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa460003520 con 0x7fa460004800 2024-05-08T06:36:42.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.389+0000 7fa475d9b640 1 --2- 172.21.15.163:0/1094438711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa460004800 0x7fa460004c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa46c001eb0 tx=0x7fa460005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27aa787432ead984 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:42.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.389+0000 7fa47559a640 1 -- 172.21.15.163:0/1094438711 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c007d10 con 0x7fa460004800 2024-05-08T06:36:42.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.389+0000 7fa47559a640 1 -- 172.21.15.163:0/1094438711 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46c005ad0 con 0x7fa460004800 2024-05-08T06:36:42.392 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.390+0000 7fa47559a640 1 -- 172.21.15.163:0/1094438711 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c007070 con 0x7fa460004800 2024-05-08T06:36:42.392 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.390+0000 7fa47ca99640 1 -- 172.21.15.163:0/1094438711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa460004800 msgr2=0x7fa460004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:42.392 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.390+0000 7fa47ca99640 1 --2- 172.21.15.163:0/1094438711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa460004800 0x7fa460004c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa46c001eb0 tx=0x7fa460005830 comp rx=0 tx=0).stop 2024-05-08T06:36:42.392 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.390+0000 7fa476d9d640 1 -- 172.21.15.163:0/1094438711 reap_dead start 2024-05-08T06:36:42.392 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.390+0000 7fa47ca99640 1 -- 172.21.15.163:0/1094438711 shutdown_connections 2024-05-08T06:36:42.392 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.390+0000 7fa47ca99640 1 -- 172.21.15.163:0/1094438711 >> 172.21.15.163:0/1094438711 conn(0x7fa47809e5e0 msgr2=0x7fa4780d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:42.392 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.390+0000 7fa47ca99640 1 -- 172.21.15.163:0/1094438711 shutdown_connections 2024-05-08T06:36:42.392 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.390+0000 7fa47ca99640 1 -- 172.21.15.163:0/1094438711 wait complete. 2024-05-08T06:36:42.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.391+0000 7fa47ca99640 1 Processor -- start 2024-05-08T06:36:42.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.392+0000 7fa47ca99640 1 -- start start 2024-05-08T06:36:42.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.392+0000 7fa47ca99640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa460004800 0x7fa4780bad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:42.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.392+0000 7fa47ca99640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4780b8770 0x7fa478162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:42.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.392+0000 7fa47ca99640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4781f8e20 0x7fa4780b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:42.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.392+0000 7fa47ca99640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa47815fb30 con 0x7fa4780b8770 2024-05-08T06:36:42.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.392+0000 7fa47ca99640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa47815f9b0 con 0x7fa4781f8e20 2024-05-08T06:36:42.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.392+0000 7fa47ca99640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa4780d5b00 con 0x7fa460004800 2024-05-08T06:36:42.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.392+0000 7fa475d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4780b8770 0x7fa478162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:42.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.392+0000 7fa475d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4780b8770 0x7fa478162160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35430/0 (socket says 172.21.15.163:35430) 2024-05-08T06:36:42.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.392+0000 7fa475d9b640 1 -- 172.21.15.163:0/2870458095 learned_addr learned my addr 172.21.15.163:0/2870458095 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:42.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.393+0000 7fa476d9d640 1 --2- 172.21.15.163:0/2870458095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4781f8e20 0x7fa4780b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:42.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.393+0000 7fa475d9b640 1 -- 172.21.15.163:0/2870458095 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa460004800 msgr2=0x7fa4780bad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.393+0000 7fa475d9b640 1 --2- 172.21.15.163:0/2870458095 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa460004800 0x7fa4780bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.393+0000 7fa475d9b640 1 -- 172.21.15.163:0/2870458095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4781f8e20 msgr2=0x7fa4780b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.393+0000 7fa475d9b640 1 --2- 172.21.15.163:0/2870458095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4781f8e20 0x7fa4780b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.393+0000 7fa475d9b640 1 -- 172.21.15.163:0/2870458095 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4780b84e0 con 0x7fa4780b8770 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.393+0000 7fa475d9b640 1 --2- 172.21.15.163:0/2870458095 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4780b8770 0x7fa478162160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00c2d0 tx=0x7fa4780d1470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.393+0000 7fa466ffd640 1 -- 172.21.15.163:0/2870458095 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c040020 con 0x7fa4780b8770 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.393+0000 7fa466ffd640 1 -- 172.21.15.163:0/2870458095 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46c0014b0 con 0x7fa4780b8770 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.393+0000 7fa47ca99640 1 -- 172.21.15.163:0/2870458095 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4781626a0 con 0x7fa4780b8770 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.393+0000 7fa466ffd640 1 -- 172.21.15.163:0/2870458095 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c037040 con 0x7fa4780b8770 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.393+0000 7fa47ca99640 1 -- 172.21.15.163:0/2870458095 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4780b8200 con 0x7fa4780b8770 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.394+0000 7fa47ca99640 1 -- 172.21.15.163:0/2870458095 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa47806a550 con 0x7fa4780b8770 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.395+0000 7fa466ffd640 1 -- 172.21.15.163:0/2870458095 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa46c043020 con 0x7fa4780b8770 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.395+0000 7fa466ffd640 1 --2- 172.21.15.163:0/2870458095 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa4580762d0 0x7fa458078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.395+0000 7fa47659c640 1 --2- 172.21.15.163:0/2870458095 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa4580762d0 0x7fa458078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:42.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.395+0000 7fa466ffd640 1 -- 172.21.15.163:0/2870458095 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fa46c0b69b0 con 0x7fa4780b8770 2024-05-08T06:36:42.397 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.396+0000 7fa47659c640 1 --2- 172.21.15.163:0/2870458095 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa4580762d0 0x7fa458078790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa4780a2660 tx=0x7fa4680095d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:42.398 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.400+0000 7fa466ffd640 1 -- 172.21.15.163:0/2870458095 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa46c0803a0 con 0x7fa4780b8770 2024-05-08T06:36:42.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:42.620+0000 7fa47ca99640 1 -- 172.21.15.163:0/2870458095 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa4780b5280 con 0x7fa4780b8770 2024-05-08T06:36:43.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.541+0000 7fa466ffd640 1 -- 172.21.15.163:0/2870458095 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7fa46c088390 con 0x7fa4780b8770 2024-05-08T06:36:43.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.544+0000 7fa47ca99640 1 -- 172.21.15.163:0/2870458095 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa4580762d0 msgr2=0x7fa458078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:43.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.544+0000 7fa47ca99640 1 --2- 172.21.15.163:0/2870458095 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa4580762d0 0x7fa458078790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa4780a2660 tx=0x7fa4680095d0 comp rx=0 tx=0).stop 2024-05-08T06:36:43.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.544+0000 7fa47ca99640 1 -- 172.21.15.163:0/2870458095 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4780b8770 msgr2=0x7fa478162160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:43.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.544+0000 7fa47ca99640 1 --2- 172.21.15.163:0/2870458095 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4780b8770 0x7fa478162160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00c2d0 tx=0x7fa4780d1470 comp rx=0 tx=0).stop 2024-05-08T06:36:43.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.544+0000 7fa47ca99640 1 -- 172.21.15.163:0/2870458095 shutdown_connections 2024-05-08T06:36:43.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.544+0000 7fa47ca99640 1 --2- 172.21.15.163:0/2870458095 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa4580762d0 0x7fa458078790 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:43.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.544+0000 7fa47ca99640 1 --2- 172.21.15.163:0/2870458095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4781f8e20 0x7fa4780b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:43.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.544+0000 7fa47ca99640 1 --2- 172.21.15.163:0/2870458095 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4780b8770 0x7fa478162160 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:43.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.544+0000 7fa47ca99640 1 --2- 172.21.15.163:0/2870458095 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa460004800 0x7fa4780bad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:43.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.544+0000 7fa47ca99640 1 -- 172.21.15.163:0/2870458095 >> 172.21.15.163:0/2870458095 conn(0x7fa47809e5e0 msgr2=0x7fa4780b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:43.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.544+0000 7fa47ca99640 1 -- 172.21.15.163:0/2870458095 shutdown_connections 2024-05-08T06:36:43.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.544+0000 7fa47ca99640 1 -- 172.21.15.163:0/2870458095 wait complete. 2024-05-08T06:36:43.556 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T06:36:43.557 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T06:36:43.665 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.666+0000 7f200f06d640 1 Processor -- start 2024-05-08T06:36:43.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.666+0000 7f200f06d640 1 -- start start 2024-05-08T06:36:43.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.667+0000 7f200f06d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2008140100 0x7f2008070670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:43.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.667+0000 7f200f06d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2008070bb0 0x7f2008160e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:43.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.667+0000 7f200f06d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20080d29d0 con 0x7f2008161340 2024-05-08T06:36:43.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.667+0000 7f200f06d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f200815d7a0 con 0x7f2008070bb0 2024-05-08T06:36:43.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.667+0000 7f200f06d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2008161b70 con 0x7f2008140100 2024-05-08T06:36:43.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.667+0000 7f2007fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2008070bb0 0x7f2008160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:43.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.667+0000 7f200d60a640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2008161340 0x7f20080b4c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:59842/0 (socket says 172.21.15.163:59842) 2024-05-08T06:36:43.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.667+0000 7f200d60a640 1 -- 172.21.15.163:0/3873711885 learned_addr learned my addr 172.21.15.163:0/3873711885 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:43.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.668+0000 7f200ce09640 1 --2- 172.21.15.163:0/3873711885 >> v2:172.21.15.192:3300/0 conn(0x7f2008140100 0x7f2008070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:43.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.668+0000 7f2007fff640 1 -- 172.21.15.163:0/3873711885 >> v2:172.21.15.192:3300/0 conn(0x7f2008140100 msgr2=0x7f2008070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:43.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.668+0000 7f2007fff640 1 --2- 172.21.15.163:0/3873711885 >> v2:172.21.15.192:3300/0 conn(0x7f2008140100 0x7f2008070670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:43.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.668+0000 7f2007fff640 1 -- 172.21.15.163:0/3873711885 >> v1:172.21.15.163:6789/0 conn(0x7f2008161340 legacy=0x7f20080b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:43.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.668+0000 7f2007fff640 1 -- 172.21.15.163:0/3873711885 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200815da60 con 0x7f2008070bb0 2024-05-08T06:36:43.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.668+0000 7f20077fe640 1 -- 172.21.15.163:0/3873711885 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4080590792 0 0) 0x7f20080d29d0 con 0x7f2008161340 2024-05-08T06:36:43.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.668+0000 7f2007fff640 1 --2- 172.21.15.163:0/3873711885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008070bb0 0x7f2008160e00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f20080d3a90 tx=0x7f1ffc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91715ae10c47c8c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:43.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.669+0000 7f20077fe640 1 -- 172.21.15.163:0/3873711885 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc007740 con 0x7f2008070bb0 2024-05-08T06:36:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.669+0000 7f20077fe640 1 -- 172.21.15.163:0/3873711885 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ffc0078e0 con 0x7f2008070bb0 2024-05-08T06:36:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.669+0000 7f20077fe640 1 -- 172.21.15.163:0/3873711885 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc007cb0 con 0x7f2008070bb0 2024-05-08T06:36:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.669+0000 7f200f06d640 1 -- 172.21.15.163:0/3873711885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008070bb0 msgr2=0x7f2008160e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.669+0000 7f200f06d640 1 --2- 172.21.15.163:0/3873711885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008070bb0 0x7f2008160e00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f20080d3a90 tx=0x7f1ffc02a5f0 comp rx=0 tx=0).stop 2024-05-08T06:36:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.669+0000 7f200f06d640 1 -- 172.21.15.163:0/3873711885 shutdown_connections 2024-05-08T06:36:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.669+0000 7f200f06d640 1 --2- 172.21.15.163:0/3873711885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008070bb0 0x7f2008160e00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.669+0000 7f200f06d640 1 --2- 172.21.15.163:0/3873711885 >> v2:172.21.15.192:3300/0 conn(0x7f2008140100 0x7f2008070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.669+0000 7f200f06d640 1 -- 172.21.15.163:0/3873711885 >> 172.21.15.163:0/3873711885 conn(0x7f2008072a60 msgr2=0x7f200806c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.669+0000 7f200f06d640 1 -- 172.21.15.163:0/3873711885 shutdown_connections 2024-05-08T06:36:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.669+0000 7f200f06d640 1 -- 172.21.15.163:0/3873711885 wait complete. 2024-05-08T06:36:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.670+0000 7f200f06d640 1 Processor -- start 2024-05-08T06:36:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.670+0000 7f200f06d640 1 -- start start 2024-05-08T06:36:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.671+0000 7f200f06d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2008070bb0 0x7f2008153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.671+0000 7f200f06d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2008140100 0x7f2008147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.671+0000 7f200f06d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008161340 0x7f200815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.671+0000 7f200f06d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f20081404e0 con 0x7f2008161340 2024-05-08T06:36:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.671+0000 7f200f06d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f200815e020 con 0x7f2008070bb0 2024-05-08T06:36:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.671+0000 7f200f06d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f20081617b0 con 0x7f2008140100 2024-05-08T06:36:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.671+0000 7f200d60a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008161340 0x7f200815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.671+0000 7f200d60a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008161340 0x7f200815c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35460/0 (socket says 172.21.15.163:35460) 2024-05-08T06:36:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.671+0000 7f200d60a640 1 -- 172.21.15.163:0/1940970547 learned_addr learned my addr 172.21.15.163:0/1940970547 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.672+0000 7f200ce09640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2008070bb0 0x7f2008153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:43.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.672+0000 7f2007fff640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2008140100 0x7f2008147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:43.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.672+0000 7f200d60a640 1 -- 172.21.15.163:0/1940970547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2008140100 msgr2=0x7f2008147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:43.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.672+0000 7f200d60a640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2008140100 0x7f2008147940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:43.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.672+0000 7f200d60a640 1 -- 172.21.15.163:0/1940970547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2008070bb0 msgr2=0x7f2008153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:43.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.672+0000 7f200d60a640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2008070bb0 0x7f2008153a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:43.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.672+0000 7f200d60a640 1 -- 172.21.15.163:0/1940970547 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200813ecf0 con 0x7f2008161340 2024-05-08T06:36:43.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.672+0000 7f2007fff640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2008140100 0x7f2008147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:43.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.672+0000 7f200d60a640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008161340 0x7f200815c230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f200000c440 tx=0x7f200000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:43.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.672+0000 7f20057fa640 1 -- 172.21.15.163:0/1940970547 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000018070 con 0x7f2008161340 2024-05-08T06:36:43.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.672+0000 7f20057fa640 1 -- 172.21.15.163:0/1940970547 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20000074e0 con 0x7f2008161340 2024-05-08T06:36:43.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.672+0000 7f200f06d640 1 -- 172.21.15.163:0/1940970547 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2008153f40 con 0x7f2008161340 2024-05-08T06:36:43.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.672+0000 7f20057fa640 1 -- 172.21.15.163:0/1940970547 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000004a20 con 0x7f2008161340 2024-05-08T06:36:43.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.672+0000 7f200f06d640 1 -- 172.21.15.163:0/1940970547 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20081484e0 con 0x7f2008161340 2024-05-08T06:36:43.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.673+0000 7f200f06d640 1 -- 172.21.15.163:0/1940970547 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20080d1010 con 0x7f2008161340 2024-05-08T06:36:43.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.674+0000 7f20057fa640 1 -- 172.21.15.163:0/1940970547 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f200001b070 con 0x7f2008161340 2024-05-08T06:36:43.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.674+0000 7f20057fa640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1ff4076300 0x7f1ff40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:43.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.674+0000 7f200ce09640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1ff4076300 0x7f1ff40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:43.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.675+0000 7f20057fa640 1 -- 172.21.15.163:0/1940970547 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f20000966c0 con 0x7f2008161340 2024-05-08T06:36:43.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.675+0000 7f200ce09640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1ff4076300 0x7f1ff40787c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f20080b7ef0 tx=0x7f1ff8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:43.678 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.679+0000 7f20057fa640 1 -- 172.21.15.163:0/1940970547 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f20080d1010 con 0x7f2008161340 2024-05-08T06:36:43.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:43.899+0000 7f200f06d640 1 -- 172.21.15.163:0/1940970547 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f200806a550 con 0x7f2008161340 2024-05-08T06:36:44.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.656+0000 7f20057fa640 1 -- 172.21.15.163:0/1940970547 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f200005f9d0 con 0x7f2008161340 2024-05-08T06:36:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.658+0000 7f200f06d640 1 -- 172.21.15.163:0/1940970547 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1ff4076300 msgr2=0x7f1ff40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.658+0000 7f200f06d640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1ff4076300 0x7f1ff40787c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f20080b7ef0 tx=0x7f1ff8008040 comp rx=0 tx=0).stop 2024-05-08T06:36:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.658+0000 7f200f06d640 1 -- 172.21.15.163:0/1940970547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008161340 msgr2=0x7f200815c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.658+0000 7f200f06d640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008161340 0x7f200815c230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f200000c440 tx=0x7f200000c900 comp rx=0 tx=0).stop 2024-05-08T06:36:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.658+0000 7f200f06d640 1 -- 172.21.15.163:0/1940970547 shutdown_connections 2024-05-08T06:36:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.658+0000 7f200f06d640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1ff4076300 0x7f1ff40787c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.658+0000 7f200f06d640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2008161340 0x7f200815c230 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.658+0000 7f200f06d640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2008140100 0x7f2008147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.658+0000 7f200f06d640 1 --2- 172.21.15.163:0/1940970547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2008070bb0 0x7f2008153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.658+0000 7f200f06d640 1 -- 172.21.15.163:0/1940970547 >> 172.21.15.163:0/1940970547 conn(0x7f2008072a60 msgr2=0x7f20080b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.659+0000 7f200f06d640 1 -- 172.21.15.163:0/1940970547 shutdown_connections 2024-05-08T06:36:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.659+0000 7f200f06d640 1 -- 172.21.15.163:0/1940970547 wait complete. 2024-05-08T06:36:44.669 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T06:36:44.669 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:36:44.769 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.770+0000 7fc3ac7da640 1 Processor -- start 2024-05-08T06:36:44.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.770+0000 7fc3ac7da640 1 -- start start 2024-05-08T06:36:44.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.771+0000 7fc3ac7da640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc3a40b0810 0x7fc3a40b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:44.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.771+0000 7fc3ac7da640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3a40741d0 con 0x7fc3a4070ea0 2024-05-08T06:36:44.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.771+0000 7fc3ac7da640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3a40b5610 con 0x7fc3a40b0430 2024-05-08T06:36:44.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.771+0000 7fc3ac7da640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc3a40b1900 con 0x7fc3a40b0810 2024-05-08T06:36:44.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.771+0000 7fc3aad77640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc3a4070ea0 0x7fc3a40b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:59852/0 (socket says 172.21.15.163:59852) 2024-05-08T06:36:44.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.771+0000 7fc3aad77640 1 -- 172.21.15.163:0/852011254 learned_addr learned my addr 172.21.15.163:0/852011254 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:44.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9d75640 1 --2- 172.21.15.163:0/852011254 >> v2:172.21.15.192:3300/0 conn(0x7fc3a40b0810 0x7fc3a40b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:44.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 -- 172.21.15.163:0/852011254 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3873892241 0 0) 0x7fc3a40741d0 con 0x7fc3a4070ea0 2024-05-08T06:36:44.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 -- 172.21.15.163:0/852011254 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc390003520 con 0x7fc3a4070ea0 2024-05-08T06:36:44.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 -- 172.21.15.163:0/852011254 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 80213135 0 0) 0x7fc390003520 con 0x7fc3a4070ea0 2024-05-08T06:36:44.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 -- 172.21.15.163:0/852011254 >> v2:172.21.15.192:3300/0 conn(0x7fc3a40b0810 msgr2=0x7fc3a40b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:44.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 --2- 172.21.15.163:0/852011254 >> v2:172.21.15.192:3300/0 conn(0x7fc3a40b0810 0x7fc3a40b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:44.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 -- 172.21.15.163:0/852011254 >> v1:172.21.15.192:6789/0 conn(0x7fc3a40b0430 legacy=0x7fc3a415f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:44.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 -- 172.21.15.163:0/852011254 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a40b6fd0 con 0x7fc3a4070ea0 2024-05-08T06:36:44.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 -- 172.21.15.163:0/852011254 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7fc3a00016f0 con 0x7fc3a4070ea0 2024-05-08T06:36:44.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 -- 172.21.15.163:0/852011254 >> v1:172.21.15.163:6789/0 conn(0x7fc3a4070ea0 legacy=0x7fc3a40b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:44.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 --2- 172.21.15.163:0/852011254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc390003a50 0x7fc390003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:44.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 --2- 172.21.15.163:0/852011254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc390004800 0x7fc390004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:44.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 --2- 172.21.15.163:0/852011254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3900052e0 0x7fc39000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:44.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 -- 172.21.15.163:0/852011254 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc39000c180 con 0x7fc390003a50 2024-05-08T06:36:44.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 -- 172.21.15.163:0/852011254 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc39000c330 con 0x7fc3900052e0 2024-05-08T06:36:44.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9574640 1 -- 172.21.15.163:0/852011254 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc39000c530 con 0x7fc390004800 2024-05-08T06:36:44.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3aa576640 1 --2- 172.21.15.163:0/852011254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc390003a50 0x7fc390003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:44.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3aad77640 1 --2- 172.21.15.163:0/852011254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3900052e0 0x7fc39000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:44.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3a9d75640 1 --2- 172.21.15.163:0/852011254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc390004800 0x7fc390004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:44.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3aa576640 1 -- 172.21.15.163:0/852011254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc390004800 msgr2=0x7fc390004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:44.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3aa576640 1 --2- 172.21.15.163:0/852011254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc390004800 0x7fc390004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:44.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3aa576640 1 -- 172.21.15.163:0/852011254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3900052e0 msgr2=0x7fc39000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:44.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3aa576640 1 --2- 172.21.15.163:0/852011254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3900052e0 0x7fc39000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:44.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3aa576640 1 -- 172.21.15.163:0/852011254 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc390003520 con 0x7fc390003a50 2024-05-08T06:36:44.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.772+0000 7fc3aad77640 1 -- 172.21.15.163:0/852011254 reap_dead start 2024-05-08T06:36:44.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.773+0000 7fc3aa576640 1 --2- 172.21.15.163:0/852011254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc390003a50 0x7fc390003e70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc39c001760 tx=0x7fc390005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d25ea7094031f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:44.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.773+0000 7fc3a9574640 1 -- 172.21.15.163:0/852011254 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0013b0 con 0x7fc390003a50 2024-05-08T06:36:44.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.773+0000 7fc3a9574640 1 -- 172.21.15.163:0/852011254 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc39c0052d0 con 0x7fc390003a50 2024-05-08T06:36:44.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.773+0000 7fc3a9574640 1 -- 172.21.15.163:0/852011254 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c02aa90 con 0x7fc390003a50 2024-05-08T06:36:44.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.774+0000 7fc3ac7da640 1 -- 172.21.15.163:0/852011254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc390003a50 msgr2=0x7fc390003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:44.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.774+0000 7fc3ac7da640 1 --2- 172.21.15.163:0/852011254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc390003a50 0x7fc390003e70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc39c001760 tx=0x7fc390005830 comp rx=0 tx=0).stop 2024-05-08T06:36:44.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.774+0000 7fc3ac7da640 1 -- 172.21.15.163:0/852011254 shutdown_connections 2024-05-08T06:36:44.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.774+0000 7fc3ac7da640 1 --2- 172.21.15.163:0/852011254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc390003a50 0x7fc390003e70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:44.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.774+0000 7fc3ac7da640 1 -- 172.21.15.163:0/852011254 >> 172.21.15.163:0/852011254 conn(0x7fc3a406d430 msgr2=0x7fc3a406d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:44.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.774+0000 7fc3ac7da640 1 -- 172.21.15.163:0/852011254 shutdown_connections 2024-05-08T06:36:44.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.774+0000 7fc3ac7da640 1 -- 172.21.15.163:0/852011254 wait complete. 2024-05-08T06:36:44.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.775+0000 7fc3ac7da640 1 Processor -- start 2024-05-08T06:36:44.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.775+0000 7fc3ac7da640 1 -- start start 2024-05-08T06:36:44.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3ac7da640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc390003a50 0x7fc3a40cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:44.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3ac7da640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3a40cc0a0 0x7fc3a40ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:44.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3ac7da640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3a40ca590 0x7fc3a40cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3ac7da640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a40b6db0 con 0x7fc3a40cc0a0 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3ac7da640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a40b6c30 con 0x7fc390003a50 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3ac7da640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a40b4fa0 con 0x7fc3a40ca590 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3a9d75640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3a40cc0a0 0x7fc3a40ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3a9d75640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3a40cc0a0 0x7fc3a40ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35478/0 (socket says 172.21.15.163:35478) 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3a9d75640 1 -- 172.21.15.163:0/2822733428 learned_addr learned my addr 172.21.15.163:0/2822733428 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3aa576640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc390003a50 0x7fc3a40cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3aad77640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3a40ca590 0x7fc3a40cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3a9d75640 1 -- 172.21.15.163:0/2822733428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3a40ca590 msgr2=0x7fc3a40cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3a9d75640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3a40ca590 0x7fc3a40cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3a9d75640 1 -- 172.21.15.163:0/2822733428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc390003a50 msgr2=0x7fc3a40cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3a9d75640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc390003a50 0x7fc3a40cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3a9d75640 1 -- 172.21.15.163:0/2822733428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a40d0d90 con 0x7fc3a40cc0a0 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3aad77640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3a40ca590 0x7fc3a40cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:44.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3a9d75640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3a40cc0a0 0x7fc3a40ca050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc398009fc0 tx=0x7fc39800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:44.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc396ffd640 1 -- 172.21.15.163:0/2822733428 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398019070 con 0x7fc3a40cc0a0 2024-05-08T06:36:44.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc396ffd640 1 -- 172.21.15.163:0/2822733428 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc398003b90 con 0x7fc3a40cc0a0 2024-05-08T06:36:44.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc396ffd640 1 -- 172.21.15.163:0/2822733428 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39800fea0 con 0x7fc3a40cc0a0 2024-05-08T06:36:44.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3ac7da640 1 -- 172.21.15.163:0/2822733428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a414e750 con 0x7fc3a40cc0a0 2024-05-08T06:36:44.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.776+0000 7fc3ac7da640 1 -- 172.21.15.163:0/2822733428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a414ed10 con 0x7fc3a40cc0a0 2024-05-08T06:36:44.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.777+0000 7fc3ac7da640 1 -- 172.21.15.163:0/2822733428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a40b3bc0 con 0x7fc3a40cc0a0 2024-05-08T06:36:44.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.778+0000 7fc396ffd640 1 -- 172.21.15.163:0/2822733428 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc39801c070 con 0x7fc3a40cc0a0 2024-05-08T06:36:44.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.778+0000 7fc396ffd640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc388076300 0x7fc3880787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:44.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.778+0000 7fc3aa576640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc388076300 0x7fc3880787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:44.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.778+0000 7fc396ffd640 1 -- 172.21.15.163:0/2822733428 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc398098100 con 0x7fc3a40cc0a0 2024-05-08T06:36:44.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.779+0000 7fc3aa576640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc388076300 0x7fc3880787c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc39c032000 tx=0x7fc39c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:44.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.783+0000 7fc396ffd640 1 -- 172.21.15.163:0/2822733428 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc3a40b3bc0 con 0x7fc3a40cc0a0 2024-05-08T06:36:44.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.999+0000 7fc3ac7da640 1 -- 172.21.15.163:0/2822733428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3a406a550 con 0x7fc3a40cc0a0 2024-05-08T06:36:44.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:44.999+0000 7fc396ffd640 1 -- 172.21.15.163:0/2822733428 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7fc398010040 con 0x7fc3a40cc0a0 2024-05-08T06:36:44.999 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:44.999 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2413407842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2413407842},{"type":"v1","addr":"172.21.15.163:6835","nonce":2413407842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2405203133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2405203133},{"type":"v1","addr":"172.21.15.163:6837","nonce":2405203133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3301087897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3301087897},{"type":"v1","addr":"172.21.15.192:6833","nonce":3301087897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:36:40.985978+0000","modified":"2024-05-08T06:36:43.900778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/354782184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":354782184},{"type":"v1","addr":"172.21.15.192:6835","nonce":354782184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-05-08T06:36:44.999 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 7 2024-05-08T06:36:45.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.001+0000 7fc3ac7da640 1 -- 172.21.15.163:0/2822733428 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc388076300 msgr2=0x7fc3880787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.001+0000 7fc3ac7da640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc388076300 0x7fc3880787c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc39c032000 tx=0x7fc39c02e000 comp rx=0 tx=0).stop 2024-05-08T06:36:45.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.001+0000 7fc3ac7da640 1 -- 172.21.15.163:0/2822733428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3a40cc0a0 msgr2=0x7fc3a40ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.001+0000 7fc3ac7da640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3a40cc0a0 0x7fc3a40ca050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc398009fc0 tx=0x7fc39800f580 comp rx=0 tx=0).stop 2024-05-08T06:36:45.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.001+0000 7fc3ac7da640 1 -- 172.21.15.163:0/2822733428 shutdown_connections 2024-05-08T06:36:45.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.001+0000 7fc3ac7da640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc388076300 0x7fc3880787c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.001+0000 7fc3ac7da640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3a40ca590 0x7fc3a40cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.001+0000 7fc3ac7da640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3a40cc0a0 0x7fc3a40ca050 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.001+0000 7fc3ac7da640 1 --2- 172.21.15.163:0/2822733428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc390003a50 0x7fc3a40cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.001+0000 7fc3ac7da640 1 -- 172.21.15.163:0/2822733428 >> 172.21.15.163:0/2822733428 conn(0x7fc3a406d430 msgr2=0x7fc3a406d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:45.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.001+0000 7fc3ac7da640 1 -- 172.21.15.163:0/2822733428 shutdown_connections 2024-05-08T06:36:45.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.001+0000 7fc3ac7da640 1 -- 172.21.15.163:0/2822733428 wait complete. 2024-05-08T06:36:45.012 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:36:45.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.112+0000 7f9a77864640 1 Processor -- start 2024-05-08T06:36:45.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.113+0000 7f9a77864640 1 -- start start 2024-05-08T06:36:45.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.113+0000 7f9a77864640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9a700b0430 0x7f9a700b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.114+0000 7f9a77864640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a700741d0 con 0x7f9a70070ea0 2024-05-08T06:36:45.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.114+0000 7f9a77864640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a700b5610 con 0x7f9a700b0810 2024-05-08T06:36:45.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.114+0000 7f9a77864640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9a700b1900 con 0x7f9a700b0430 2024-05-08T06:36:45.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.114+0000 7f9a75e01640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f9a70070ea0 0x7f9a700b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:59860/0 (socket says 172.21.15.163:59860) 2024-05-08T06:36:45.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.114+0000 7f9a75e01640 1 -- 172.21.15.163:0/281684529 learned_addr learned my addr 172.21.15.163:0/281684529 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:45.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.114+0000 7f9a75600640 1 --2- 172.21.15.163:0/281684529 >> v2:172.21.15.192:3300/0 conn(0x7f9a700b0430 0x7f9a700b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.114+0000 7f9a67fff640 1 -- 172.21.15.163:0/281684529 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1778871175 0 0) 0x7f9a700741d0 con 0x7f9a70070ea0 2024-05-08T06:36:45.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.114+0000 7f9a67fff640 1 -- 172.21.15.163:0/281684529 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a5c003520 con 0x7f9a70070ea0 2024-05-08T06:36:45.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.114+0000 7f9a75600640 1 -- 172.21.15.163:0/281684529 >> v1:172.21.15.192:6789/0 conn(0x7f9a700b0810 legacy=0x7f9a7015f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.114+0000 7f9a67fff640 1 -- 172.21.15.163:0/281684529 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4055998740 0 0) 0x7f9a700b5610 con 0x7f9a700b0810 2024-05-08T06:36:45.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.114+0000 7f9a75600640 1 -- 172.21.15.163:0/281684529 >> v1:172.21.15.163:6789/0 conn(0x7f9a70070ea0 legacy=0x7f9a700b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.114+0000 7f9a75600640 1 -- 172.21.15.163:0/281684529 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a700b6fd0 con 0x7f9a700b0430 2024-05-08T06:36:45.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.115+0000 7f9a75600640 1 --2- 172.21.15.163:0/281684529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a700b0430 0x7f9a700b4fa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0097a0 tx=0x7f9a6c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1164b77010fba9de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:45.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.115+0000 7f9a67fff640 1 -- 172.21.15.163:0/281684529 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c037070 con 0x7f9a700b0430 2024-05-08T06:36:45.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.115+0000 7f9a67fff640 1 -- 172.21.15.163:0/281684529 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a6c032440 con 0x7f9a700b0430 2024-05-08T06:36:45.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.115+0000 7f9a67fff640 1 -- 172.21.15.163:0/281684529 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0327f0 con 0x7f9a700b0430 2024-05-08T06:36:45.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.116+0000 7f9a77864640 1 -- 172.21.15.163:0/281684529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a700b0430 msgr2=0x7f9a700b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.116+0000 7f9a77864640 1 --2- 172.21.15.163:0/281684529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a700b0430 0x7f9a700b4fa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0097a0 tx=0x7f9a6c02a1a0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.116+0000 7f9a77864640 1 -- 172.21.15.163:0/281684529 shutdown_connections 2024-05-08T06:36:45.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.116+0000 7f9a77864640 1 --2- 172.21.15.163:0/281684529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a700b0430 0x7f9a700b4fa0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.116+0000 7f9a77864640 1 -- 172.21.15.163:0/281684529 >> 172.21.15.163:0/281684529 conn(0x7f9a7006d430 msgr2=0x7f9a7006d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:45.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.116+0000 7f9a77864640 1 -- 172.21.15.163:0/281684529 shutdown_connections 2024-05-08T06:36:45.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.116+0000 7f9a77864640 1 -- 172.21.15.163:0/281684529 wait complete. 2024-05-08T06:36:45.118 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.117+0000 7f9a77864640 1 Processor -- start 2024-05-08T06:36:45.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.117+0000 7f9a77864640 1 -- start start 2024-05-08T06:36:45.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a77864640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a70070ea0 0x7f9a70152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a77864640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a700b0430 0x7f9a701f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a77864640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a700b0810 0x7f9a700c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a77864640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9a700b6db0 con 0x7f9a700b0810 2024-05-08T06:36:45.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a77864640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a700b6c30 con 0x7f9a700b0430 2024-05-08T06:36:45.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a77864640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9a700b4340 con 0x7f9a70070ea0 2024-05-08T06:36:45.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a75e01640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a700b0810 0x7f9a700c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a75e01640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a700b0810 0x7f9a700c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35494/0 (socket says 172.21.15.163:35494) 2024-05-08T06:36:45.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a75e01640 1 -- 172.21.15.163:0/3300243990 learned_addr learned my addr 172.21.15.163:0/3300243990 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:45.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a74dff640 1 --2- 172.21.15.163:0/3300243990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a700b0430 0x7f9a701f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a75600640 1 --2- 172.21.15.163:0/3300243990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a70070ea0 0x7f9a70152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a75e01640 1 -- 172.21.15.163:0/3300243990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a70070ea0 msgr2=0x7f9a70152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a75e01640 1 --2- 172.21.15.163:0/3300243990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a70070ea0 0x7f9a70152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.120 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a75e01640 1 -- 172.21.15.163:0/3300243990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a700b0430 msgr2=0x7f9a701f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.120 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a75e01640 1 --2- 172.21.15.163:0/3300243990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a700b0430 0x7f9a701f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.120 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a75e01640 1 -- 172.21.15.163:0/3300243990 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a70152fb0 con 0x7f9a700b0810 2024-05-08T06:36:45.120 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a75e01640 1 --2- 172.21.15.163:0/3300243990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a700b0810 0x7f9a700c87b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800cb70 tx=0x7f9a6800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:45.120 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.118+0000 7f9a65ffb640 1 -- 172.21.15.163:0/3300243990 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800ad40 con 0x7f9a700b0810 2024-05-08T06:36:45.120 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.119+0000 7f9a65ffb640 1 -- 172.21.15.163:0/3300243990 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a68012400 con 0x7f9a700b0810 2024-05-08T06:36:45.120 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.119+0000 7f9a77864640 1 -- 172.21.15.163:0/3300243990 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a700ceb00 con 0x7f9a700b0810 2024-05-08T06:36:45.120 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.119+0000 7f9a65ffb640 1 -- 172.21.15.163:0/3300243990 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a680127b0 con 0x7f9a700b0810 2024-05-08T06:36:45.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.119+0000 7f9a77864640 1 -- 172.21.15.163:0/3300243990 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a700cf0c0 con 0x7f9a700b0810 2024-05-08T06:36:45.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.120+0000 7f9a77864640 1 -- 172.21.15.163:0/3300243990 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a7006a550 con 0x7f9a700b0810 2024-05-08T06:36:45.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.120+0000 7f9a65ffb640 1 -- 172.21.15.163:0/3300243990 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9a68012950 con 0x7f9a700b0810 2024-05-08T06:36:45.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.121+0000 7f9a65ffb640 1 --2- 172.21.15.163:0/3300243990 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9a540762d0 0x7f9a54078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.121+0000 7f9a75600640 1 --2- 172.21.15.163:0/3300243990 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9a540762d0 0x7f9a54078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.121+0000 7f9a65ffb640 1 -- 172.21.15.163:0/3300243990 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9a68097b90 con 0x7f9a700b0810 2024-05-08T06:36:45.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.121+0000 7f9a75600640 1 --2- 172.21.15.163:0/3300243990 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9a540762d0 0x7f9a54078790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c034000 tx=0x7f9a6c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:45.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.125+0000 7f9a65ffb640 1 -- 172.21.15.163:0/3300243990 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9a70152fb0 con 0x7f9a700b0810 2024-05-08T06:36:45.305 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.306+0000 7f9a77864640 1 -- 172.21.15.163:0/3300243990 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a701627b0 con 0x7f9a700b0810 2024-05-08T06:36:45.306 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.307+0000 7f9a65ffb640 1 -- 172.21.15.163:0/3300243990 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f9a68060e20 con 0x7f9a700b0810 2024-05-08T06:36:45.307 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:45.307 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:36:42.641260+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:36:38.853461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:36:39.952926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:36:45.309 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.309+0000 7f9a77864640 1 -- 172.21.15.163:0/3300243990 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9a540762d0 msgr2=0x7f9a54078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.309 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.309+0000 7f9a77864640 1 --2- 172.21.15.163:0/3300243990 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9a540762d0 0x7f9a54078790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c034000 tx=0x7f9a6c02e000 comp rx=0 tx=0).stop 2024-05-08T06:36:45.309 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.309+0000 7f9a77864640 1 -- 172.21.15.163:0/3300243990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a700b0810 msgr2=0x7f9a700c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.309 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.309+0000 7f9a77864640 1 --2- 172.21.15.163:0/3300243990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a700b0810 0x7f9a700c87b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800cb70 tx=0x7f9a6800a430 comp rx=0 tx=0).stop 2024-05-08T06:36:45.309 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.310+0000 7f9a77864640 1 -- 172.21.15.163:0/3300243990 shutdown_connections 2024-05-08T06:36:45.309 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.310+0000 7f9a77864640 1 --2- 172.21.15.163:0/3300243990 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9a540762d0 0x7f9a54078790 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.309 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.310+0000 7f9a77864640 1 --2- 172.21.15.163:0/3300243990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a700b0810 0x7f9a700c87b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.310 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.310+0000 7f9a77864640 1 --2- 172.21.15.163:0/3300243990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a700b0430 0x7f9a701f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.310 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.310+0000 7f9a77864640 1 --2- 172.21.15.163:0/3300243990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a70070ea0 0x7f9a70152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.310 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.310+0000 7f9a77864640 1 -- 172.21.15.163:0/3300243990 >> 172.21.15.163:0/3300243990 conn(0x7f9a7006d430 msgr2=0x7f9a70070290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:45.310 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.310+0000 7f9a77864640 1 -- 172.21.15.163:0/3300243990 shutdown_connections 2024-05-08T06:36:45.310 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.310+0000 7f9a77864640 1 -- 172.21.15.163:0/3300243990 wait complete. 2024-05-08T06:36:45.320 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T06:36:45.320 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:36:45.420 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.421+0000 7f0993859640 1 Processor -- start 2024-05-08T06:36:45.420 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.421+0000 7f0993859640 1 -- start start 2024-05-08T06:36:45.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f0993859640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f098c15b8a0 0x7f098c0725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f0993859640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f098c0b54d0 0x7f098c0b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f0993859640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f098c0b5140 con 0x7f098c0b46e0 2024-05-08T06:36:45.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f0993859640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f098c0af910 con 0x7f098c0b54d0 2024-05-08T06:36:45.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f0993859640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f098c0b4b80 con 0x7f098c15b8a0 2024-05-08T06:36:45.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f0990df4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f098c0b54d0 0x7f098c0b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f0991df6640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f098c0b46e0 0x7f098c06f090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:59870/0 (socket says 172.21.15.163:59870) 2024-05-08T06:36:45.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f0990df4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f098c0b54d0 0x7f098c0b41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35500/0 (socket says 172.21.15.163:35500) 2024-05-08T06:36:45.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f0991df6640 1 -- 172.21.15.163:0/4263068819 learned_addr learned my addr 172.21.15.163:0/4263068819 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:45.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f09915f5640 1 --2- 172.21.15.163:0/4263068819 >> v2:172.21.15.192:3300/0 conn(0x7f098c15b8a0 0x7f098c0725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f0990df4640 1 -- 172.21.15.163:0/4263068819 >> v2:172.21.15.192:3300/0 conn(0x7f098c15b8a0 msgr2=0x7f098c0725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:45.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f0990df4640 1 --2- 172.21.15.163:0/4263068819 >> v2:172.21.15.192:3300/0 conn(0x7f098c15b8a0 0x7f098c0725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f0990df4640 1 -- 172.21.15.163:0/4263068819 >> v1:172.21.15.163:6789/0 conn(0x7f098c0b46e0 legacy=0x7f098c06f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f0990df4640 1 -- 172.21.15.163:0/4263068819 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098c0b4e60 con 0x7f098c0b54d0 2024-05-08T06:36:45.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.422+0000 7f0983fff640 1 -- 172.21.15.163:0/4263068819 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3958315051 0 0) 0x7f098c0b5140 con 0x7f098c0b46e0 2024-05-08T06:36:45.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.423+0000 7f0990df4640 1 --2- 172.21.15.163:0/4263068819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c0b54d0 0x7f098c0b41a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f098c06d270 tx=0x7f097c02c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db34dadee94ad2cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:45.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.423+0000 7f0983fff640 1 -- 172.21.15.163:0/4263068819 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c032770 con 0x7f098c0b54d0 2024-05-08T06:36:45.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.423+0000 7f0983fff640 1 -- 172.21.15.163:0/4263068819 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f097c038070 con 0x7f098c0b54d0 2024-05-08T06:36:45.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.423+0000 7f0983fff640 1 -- 172.21.15.163:0/4263068819 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c032b30 con 0x7f098c0b54d0 2024-05-08T06:36:45.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.424+0000 7f0993859640 1 -- 172.21.15.163:0/4263068819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c0b54d0 msgr2=0x7f098c0b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.424+0000 7f0993859640 1 --2- 172.21.15.163:0/4263068819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c0b54d0 0x7f098c0b41a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f098c06d270 tx=0x7f097c02c870 comp rx=0 tx=0).stop 2024-05-08T06:36:45.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.424+0000 7f0993859640 1 -- 172.21.15.163:0/4263068819 shutdown_connections 2024-05-08T06:36:45.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.424+0000 7f0993859640 1 --2- 172.21.15.163:0/4263068819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c0b54d0 0x7f098c0b41a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.424+0000 7f0993859640 1 --2- 172.21.15.163:0/4263068819 >> v2:172.21.15.192:3300/0 conn(0x7f098c15b8a0 0x7f098c0725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.424+0000 7f0993859640 1 -- 172.21.15.163:0/4263068819 >> 172.21.15.163:0/4263068819 conn(0x7f098c06c500 msgr2=0x7f098c06c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:45.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.424+0000 7f0993859640 1 -- 172.21.15.163:0/4263068819 shutdown_connections 2024-05-08T06:36:45.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.424+0000 7f0993859640 1 -- 172.21.15.163:0/4263068819 wait complete. 2024-05-08T06:36:45.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.425+0000 7f0993859640 1 Processor -- start 2024-05-08T06:36:45.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.425+0000 7f0993859640 1 -- start start 2024-05-08T06:36:45.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f0993859640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c0b46e0 0x7f098c150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f0993859640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098c0b54d0 0x7f098c15f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f0993859640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f098c15b8a0 0x7f098c151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f0993859640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f098c072c60 con 0x7f098c0b46e0 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f0993859640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f098c072ae0 con 0x7f098c0b54d0 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f0993859640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f098c06f7a0 con 0x7f098c15b8a0 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f09915f5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c0b46e0 0x7f098c150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f09915f5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c0b46e0 0x7f098c150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35512/0 (socket says 172.21.15.163:35512) 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f09915f5640 1 -- 172.21.15.163:0/2546947727 learned_addr learned my addr 172.21.15.163:0/2546947727 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f0990df4640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098c0b54d0 0x7f098c15f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f0991df6640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f098c15b8a0 0x7f098c151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f09915f5640 1 -- 172.21.15.163:0/2546947727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f098c15b8a0 msgr2=0x7f098c151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f09915f5640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f098c15b8a0 0x7f098c151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f09915f5640 1 -- 172.21.15.163:0/2546947727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098c0b54d0 msgr2=0x7f098c15f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f09915f5640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098c0b54d0 0x7f098c15f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f09915f5640 1 -- 172.21.15.163:0/2546947727 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098c154350 con 0x7f098c0b46e0 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f0990df4640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098c0b54d0 0x7f098c15f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f0991df6640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f098c15b8a0 0x7f098c151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:45.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f09915f5640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c0b46e0 0x7f098c150960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f098800e870 tx=0x7f098800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:45.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f0981ffb640 1 -- 172.21.15.163:0/2546947727 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988017070 con 0x7f098c0b46e0 2024-05-08T06:36:45.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f0981ffb640 1 -- 172.21.15.163:0/2546947727 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f098800c940 con 0x7f098c0b46e0 2024-05-08T06:36:45.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f0981ffb640 1 -- 172.21.15.163:0/2546947727 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098800cd30 con 0x7f098c0b46e0 2024-05-08T06:36:45.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.426+0000 7f0993859640 1 -- 172.21.15.163:0/2546947727 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f098c154070 con 0x7f098c0b46e0 2024-05-08T06:36:45.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.427+0000 7f0993859640 1 -- 172.21.15.163:0/2546947727 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f098c150160 con 0x7f098c0b46e0 2024-05-08T06:36:45.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.428+0000 7f0993859640 1 -- 172.21.15.163:0/2546947727 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f098c06a550 con 0x7f098c0b46e0 2024-05-08T06:36:45.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.428+0000 7f0981ffb640 1 -- 172.21.15.163:0/2546947727 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f098801a070 con 0x7f098c0b46e0 2024-05-08T06:36:45.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.429+0000 7f0981ffb640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f09700762d0 0x7f0970078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.429+0000 7f0990df4640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f09700762d0 0x7f0970078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.429+0000 7f0981ffb640 1 -- 172.21.15.163:0/2546947727 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0988097430 con 0x7f098c0b46e0 2024-05-08T06:36:45.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.429+0000 7f0990df4640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f09700762d0 0x7f0970078790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f097c037000 tx=0x7f097c032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:45.432 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.433+0000 7f0981ffb640 1 -- 172.21.15.163:0/2546947727 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0988060e20 con 0x7f098c0b46e0 2024-05-08T06:36:45.613 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.614+0000 7f0993859640 1 -- 172.21.15.163:0/2546947727 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f098c0b5140 con 0x7f09700762d0 2024-05-08T06:36:45.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.620+0000 7f0981ffb640 1 -- 172.21.15.163:0/2546947727 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237066 (secure 0 0 0) 0x7f098c0b5140 con 0x7f09700762d0 2024-05-08T06:36:45.621 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:45.623 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2024-05-08T06:36:45.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.623+0000 7f0993859640 1 -- 172.21.15.163:0/2546947727 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f09700762d0 msgr2=0x7f0970078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.623+0000 7f0993859640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f09700762d0 0x7f0970078790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f097c037000 tx=0x7f097c032000 comp rx=0 tx=0).stop 2024-05-08T06:36:45.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.623+0000 7f0993859640 1 -- 172.21.15.163:0/2546947727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c0b46e0 msgr2=0x7f098c150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.623+0000 7f0993859640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c0b46e0 0x7f098c150960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f098800e870 tx=0x7f098800ed30 comp rx=0 tx=0).stop 2024-05-08T06:36:45.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.623+0000 7f0993859640 1 -- 172.21.15.163:0/2546947727 shutdown_connections 2024-05-08T06:36:45.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.623+0000 7f0993859640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f09700762d0 0x7f0970078790 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.623+0000 7f0993859640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f098c15b8a0 0x7f098c151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.623+0000 7f0993859640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098c0b54d0 0x7f098c15f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.623+0000 7f0993859640 1 --2- 172.21.15.163:0/2546947727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f098c0b46e0 0x7f098c150960 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.623+0000 7f0993859640 1 -- 172.21.15.163:0/2546947727 >> 172.21.15.163:0/2546947727 conn(0x7f098c06c500 msgr2=0x7f098c06c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:45.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.623+0000 7f0993859640 1 -- 172.21.15.163:0/2546947727 shutdown_connections 2024-05-08T06:36:45.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.623+0000 7f0993859640 1 -- 172.21.15.163:0/2546947727 wait complete. 2024-05-08T06:36:45.633 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-08T06:36:44.097893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219060,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755660,"statfs":{"total":773094113280,"available":772869795840,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.473394"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":102,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T06:36:32.805463+0000","last_change":"2024-05-08T06:36:31.710287+0000","last_active":"2024-05-08T06:36:32.805463+0000","last_peered":"2024-05-08T06:36:32.805463+0000","last_clean":"2024-05-08T06:36:32.805463+0000","last_became_active":"2024-05-08T06:36:31.709270+0000","last_became_peered":"2024-05-08T06:36:31.709270+0000","last_unstale":"2024-05-08T06:36:32.805463+0000","last_undegraded":"2024-05-08T06:36:32.805463+0000","last_fullsized":"2024-05-08T06:36:32.805463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_clean_scrub_stamp":"2024-05-08T06:36:30.122918+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:42.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:36:45.637 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:36:45.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.775+0000 7f2677ba4640 1 Processor -- start 2024-05-08T06:36:45.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.775+0000 7f2677ba4640 1 -- start start 2024-05-08T06:36:45.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.776+0000 7f2677ba4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2670142390 0x7f26700b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.776+0000 7f2677ba4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f26700b65e0 0x7f2670162fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.776+0000 7f2677ba4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2670142770 con 0x7f2670163500 2024-05-08T06:36:45.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.776+0000 7f2677ba4640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f26700b6a40 con 0x7f26700b65e0 2024-05-08T06:36:45.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.776+0000 7f2677ba4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2670143a80 con 0x7f2670142390 2024-05-08T06:36:45.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.776+0000 7f267513f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f26700b65e0 0x7f2670162fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.776+0000 7f2676141640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f2670163500 0x7f2670071db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:34050/0 (socket says 172.21.15.163:34050) 2024-05-08T06:36:45.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.776+0000 7f2675940640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2670142390 0x7f26700b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.776+0000 7f2676141640 1 -- 172.21.15.163:0/106286019 learned_addr learned my addr 172.21.15.163:0/106286019 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:45.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.777+0000 7f267513f640 1 -- 172.21.15.163:0/106286019 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2670142390 msgr2=0x7f26700b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:45.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.777+0000 7f267513f640 1 --2- 172.21.15.163:0/106286019 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2670142390 0x7f26700b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.777+0000 7f267513f640 1 -- 172.21.15.163:0/106286019 >> v1:172.21.15.192:6789/0 conn(0x7f2670163500 legacy=0x7f2670071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.777+0000 7f267493e640 1 -- 172.21.15.163:0/106286019 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 276492819 0 0) 0x7f2670142770 con 0x7f2670163500 2024-05-08T06:36:45.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.777+0000 7f267513f640 1 -- 172.21.15.163:0/106286019 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2670163d30 con 0x7f26700b65e0 2024-05-08T06:36:45.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.777+0000 7f2675940640 1 --2- 172.21.15.163:0/106286019 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2670142390 0x7f26700b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:45.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.778+0000 7f267513f640 1 --2- 172.21.15.163:0/106286019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26700b65e0 0x7f2670162fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f26600097a0 tx=0x7f266002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38aab28b3ab5bc40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:45.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.778+0000 7f267493e640 1 -- 172.21.15.163:0/106286019 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660037070 con 0x7f26700b65e0 2024-05-08T06:36:45.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.778+0000 7f267493e640 1 -- 172.21.15.163:0/106286019 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2660032440 con 0x7f26700b65e0 2024-05-08T06:36:45.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.778+0000 7f267493e640 1 -- 172.21.15.163:0/106286019 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660032810 con 0x7f26700b65e0 2024-05-08T06:36:45.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.778+0000 7f2677ba4640 1 -- 172.21.15.163:0/106286019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26700b65e0 msgr2=0x7f2670162fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.778+0000 7f2677ba4640 1 --2- 172.21.15.163:0/106286019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26700b65e0 0x7f2670162fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f26600097a0 tx=0x7f266002a180 comp rx=0 tx=0).stop 2024-05-08T06:36:45.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.778+0000 7f2677ba4640 1 -- 172.21.15.163:0/106286019 shutdown_connections 2024-05-08T06:36:45.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.778+0000 7f2677ba4640 1 --2- 172.21.15.163:0/106286019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26700b65e0 0x7f2670162fc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.778+0000 7f2677ba4640 1 --2- 172.21.15.163:0/106286019 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2670142390 0x7f26700b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.778+0000 7f2677ba4640 1 -- 172.21.15.163:0/106286019 >> 172.21.15.163:0/106286019 conn(0x7f26700b4da0 msgr2=0x7f26700b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:45.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.778+0000 7f2677ba4640 1 -- 172.21.15.163:0/106286019 shutdown_connections 2024-05-08T06:36:45.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.778+0000 7f2677ba4640 1 -- 172.21.15.163:0/106286019 wait complete. 2024-05-08T06:36:45.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.780+0000 7f2677ba4640 1 Processor -- start 2024-05-08T06:36:45.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.780+0000 7f2677ba4640 1 -- start start 2024-05-08T06:36:45.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.780+0000 7f2677ba4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26700b65e0 0x7f267015caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.780+0000 7f2677ba4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2670142390 0x7f267006ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.780+0000 7f2677ba4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2670163500 0x7f2670161400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.780+0000 7f2677ba4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f267015ff50 con 0x7f2670163500 2024-05-08T06:36:45.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.780+0000 7f2677ba4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f267015fc70 con 0x7f26700b65e0 2024-05-08T06:36:45.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.780+0000 7f2677ba4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2670160230 con 0x7f2670142390 2024-05-08T06:36:45.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.780+0000 7f2676141640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2670163500 0x7f2670161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.780+0000 7f2676141640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2670163500 0x7f2670161400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35526/0 (socket says 172.21.15.163:35526) 2024-05-08T06:36:45.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.780+0000 7f2676141640 1 -- 172.21.15.163:0/2309123036 learned_addr learned my addr 172.21.15.163:0/2309123036 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:45.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.781+0000 7f2675940640 1 --2- 172.21.15.163:0/2309123036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26700b65e0 0x7f267015caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.781+0000 7f267513f640 1 --2- 172.21.15.163:0/2309123036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2670142390 0x7f267006ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.781+0000 7f2676141640 1 -- 172.21.15.163:0/2309123036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2670142390 msgr2=0x7f267006ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.781+0000 7f2676141640 1 --2- 172.21.15.163:0/2309123036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2670142390 0x7f267006ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.781+0000 7f2676141640 1 -- 172.21.15.163:0/2309123036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26700b65e0 msgr2=0x7f267015caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.781+0000 7f2676141640 1 --2- 172.21.15.163:0/2309123036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26700b65e0 0x7f267015caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.781+0000 7f2676141640 1 -- 172.21.15.163:0/2309123036 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267015cfe0 con 0x7f2670163500 2024-05-08T06:36:45.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.781+0000 7f2676141640 1 --2- 172.21.15.163:0/2309123036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2670163500 0x7f2670161400 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f266800c7a0 tx=0x7f266800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:45.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.781+0000 7f26667fc640 1 -- 172.21.15.163:0/2309123036 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2668018070 con 0x7f2670163500 2024-05-08T06:36:45.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.781+0000 7f26667fc640 1 -- 172.21.15.163:0/2309123036 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f266800a5e0 con 0x7f2670163500 2024-05-08T06:36:45.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.781+0000 7f2677ba4640 1 -- 172.21.15.163:0/2309123036 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267006db30 con 0x7f2670163500 2024-05-08T06:36:45.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.781+0000 7f26667fc640 1 -- 172.21.15.163:0/2309123036 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266800aa10 con 0x7f2670163500 2024-05-08T06:36:45.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.781+0000 7f2677ba4640 1 -- 172.21.15.163:0/2309123036 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267006e0f0 con 0x7f2670163500 2024-05-08T06:36:45.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.782+0000 7f2677ba4640 1 -- 172.21.15.163:0/2309123036 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267006a550 con 0x7f2670163500 2024-05-08T06:36:45.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.783+0000 7f26667fc640 1 -- 172.21.15.163:0/2309123036 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f266801b070 con 0x7f2670163500 2024-05-08T06:36:45.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.783+0000 7f26667fc640 1 --2- 172.21.15.163:0/2309123036 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f26540762d0 0x7f2654078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:45.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.783+0000 7f26667fc640 1 -- 172.21.15.163:0/2309123036 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2668096a60 con 0x7f2670163500 2024-05-08T06:36:45.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.783+0000 7f2675940640 1 --2- 172.21.15.163:0/2309123036 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f26540762d0 0x7f2654078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:45.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.784+0000 7f2675940640 1 --2- 172.21.15.163:0/2309123036 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f26540762d0 0x7f2654078790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f266c009560 tx=0x7f266c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:45.786 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.788+0000 7f26667fc640 1 -- 172.21.15.163:0/2309123036 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f266805fe00 con 0x7f2670163500 2024-05-08T06:36:45.963 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.964+0000 7f2677ba4640 1 -- 172.21.15.163:0/2309123036 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2670072ab0 con 0x7f26540762d0 2024-05-08T06:36:45.968 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.969+0000 7f26667fc640 1 -- 172.21.15.163:0/2309123036 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237066 (secure 0 0 0) 0x7f2670072ab0 con 0x7f26540762d0 2024-05-08T06:36:45.971 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:45.972 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2024-05-08T06:36:45.973 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.972+0000 7f2677ba4640 1 -- 172.21.15.163:0/2309123036 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f26540762d0 msgr2=0x7f2654078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.973 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.972+0000 7f2677ba4640 1 --2- 172.21.15.163:0/2309123036 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f26540762d0 0x7f2654078790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f266c009560 tx=0x7f266c009190 comp rx=0 tx=0).stop 2024-05-08T06:36:45.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.972+0000 7f2677ba4640 1 -- 172.21.15.163:0/2309123036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2670163500 msgr2=0x7f2670161400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:45.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.972+0000 7f2677ba4640 1 --2- 172.21.15.163:0/2309123036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2670163500 0x7f2670161400 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f266800c7a0 tx=0x7f266800cc60 comp rx=0 tx=0).stop 2024-05-08T06:36:45.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.972+0000 7f2677ba4640 1 -- 172.21.15.163:0/2309123036 shutdown_connections 2024-05-08T06:36:45.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.972+0000 7f2677ba4640 1 --2- 172.21.15.163:0/2309123036 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f26540762d0 0x7f2654078790 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.972+0000 7f2677ba4640 1 --2- 172.21.15.163:0/2309123036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2670163500 0x7f2670161400 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.972+0000 7f2677ba4640 1 --2- 172.21.15.163:0/2309123036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2670142390 0x7f267006ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.972+0000 7f2677ba4640 1 --2- 172.21.15.163:0/2309123036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26700b65e0 0x7f267015caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:45.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.972+0000 7f2677ba4640 1 -- 172.21.15.163:0/2309123036 >> 172.21.15.163:0/2309123036 conn(0x7f26700b4da0 msgr2=0x7f2670144da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:45.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.972+0000 7f2677ba4640 1 -- 172.21.15.163:0/2309123036 shutdown_connections 2024-05-08T06:36:45.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:45.972+0000 7f2677ba4640 1 -- 172.21.15.163:0/2309123036 wait complete. 2024-05-08T06:36:45.983 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-08T06:36:44.097893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219060,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755660,"statfs":{"total":773094113280,"available":772869795840,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.473394"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":102,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T06:36:32.805463+0000","last_change":"2024-05-08T06:36:31.710287+0000","last_active":"2024-05-08T06:36:32.805463+0000","last_peered":"2024-05-08T06:36:32.805463+0000","last_clean":"2024-05-08T06:36:32.805463+0000","last_became_active":"2024-05-08T06:36:31.709270+0000","last_became_peered":"2024-05-08T06:36:31.709270+0000","last_unstale":"2024-05-08T06:36:32.805463+0000","last_undegraded":"2024-05-08T06:36:32.805463+0000","last_fullsized":"2024-05-08T06:36:32.805463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_clean_scrub_stamp":"2024-05-08T06:36:30.122918+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:42.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:40.623604+0000","last_change":"2024-05-08T06:36:40.623604+0000","last_active":"2024-05-08T06:36:40.623604+0000","last_peered":"2024-05-08T06:36:40.623604+0000","last_clean":"2024-05-08T06:36:40.623604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:40.623604+0000","last_undegraded":"2024-05-08T06:36:40.623604+0000","last_fullsized":"2024-05-08T06:36:40.623604+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:36:39.618012+0000","last_change":"2024-05-08T06:36:39.618012+0000","last_active":"2024-05-08T06:36:39.618012+0000","last_peered":"2024-05-08T06:36:39.618012+0000","last_clean":"2024-05-08T06:36:39.618012+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:36:39.618012+0000","last_undegraded":"2024-05-08T06:36:39.618012+0000","last_fullsized":"2024-05-08T06:36:39.618012+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:36:45.986 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:36:46.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.124+0000 7faa88bac640 1 Processor -- start 2024-05-08T06:36:46.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.124+0000 7faa88bac640 1 -- start start 2024-05-08T06:36:46.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.125+0000 7faa88bac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faa840ae180 0x7faa840b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.125+0000 7faa88bac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa84072f80 con 0x7faa8415c020 2024-05-08T06:36:46.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.125+0000 7faa88bac640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa84074430 con 0x7faa840b46e0 2024-05-08T06:36:46.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.125+0000 7faa88bac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7faa840ae700 con 0x7faa840ae180 2024-05-08T06:36:46.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.125+0000 7faa8259c640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7faa8415c020 0x7faa84073720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:59880/0 (socket says 172.21.15.163:59880) 2024-05-08T06:36:46.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.125+0000 7faa81d9b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faa840ae180 0x7faa840b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.125+0000 7faa8259c640 1 -- 172.21.15.163:0/3122111513 learned_addr learned my addr 172.21.15.163:0/3122111513 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:46.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.126+0000 7faa81d9b640 1 -- 172.21.15.163:0/3122111513 >> v1:172.21.15.192:6789/0 conn(0x7faa840b46e0 legacy=0x7faa840afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:46.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.126+0000 7faa81d9b640 1 -- 172.21.15.163:0/3122111513 >> v1:172.21.15.163:6789/0 conn(0x7faa8415c020 legacy=0x7faa84073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.126+0000 7faa81d9b640 1 -- 172.21.15.163:0/3122111513 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa84074710 con 0x7faa840ae180 2024-05-08T06:36:46.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.126+0000 7faa81d9b640 1 --2- 172.21.15.163:0/3122111513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa840ae180 0x7faa840b41a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faa780097a0 tx=0x7faa7802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8c482167766032 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:46.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.126+0000 7faa8159a640 1 -- 172.21.15.163:0/3122111513 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78037070 con 0x7faa840ae180 2024-05-08T06:36:46.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.126+0000 7faa8159a640 1 -- 172.21.15.163:0/3122111513 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa7802adb0 con 0x7faa840ae180 2024-05-08T06:36:46.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.126+0000 7faa8159a640 1 -- 172.21.15.163:0/3122111513 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78032660 con 0x7faa840ae180 2024-05-08T06:36:46.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.127+0000 7faa88bac640 1 -- 172.21.15.163:0/3122111513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa840ae180 msgr2=0x7faa840b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.127+0000 7faa88bac640 1 --2- 172.21.15.163:0/3122111513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa840ae180 0x7faa840b41a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faa780097a0 tx=0x7faa7802a060 comp rx=0 tx=0).stop 2024-05-08T06:36:46.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.127+0000 7faa88bac640 1 -- 172.21.15.163:0/3122111513 shutdown_connections 2024-05-08T06:36:46.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.127+0000 7faa88bac640 1 --2- 172.21.15.163:0/3122111513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa840ae180 0x7faa840b41a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.127+0000 7faa88bac640 1 -- 172.21.15.163:0/3122111513 >> 172.21.15.163:0/3122111513 conn(0x7faa8406c500 msgr2=0x7faa8406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:46.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.127+0000 7faa88bac640 1 -- 172.21.15.163:0/3122111513 shutdown_connections 2024-05-08T06:36:46.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.127+0000 7faa88bac640 1 -- 172.21.15.163:0/3122111513 wait complete. 2024-05-08T06:36:46.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.128+0000 7faa88bac640 1 Processor -- start 2024-05-08T06:36:46.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.128+0000 7faa88bac640 1 -- start start 2024-05-08T06:36:46.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa88bac640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa840ae180 0x7faa8414d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa88bac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa840b46e0 0x7faa8414dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa88bac640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa8415c020 0x7faa8414e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa88bac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7faa8415c400 con 0x7faa840b46e0 2024-05-08T06:36:46.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa88bac640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faa84073290 con 0x7faa840ae180 2024-05-08T06:36:46.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa88bac640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7faa840b4b70 con 0x7faa8415c020 2024-05-08T06:36:46.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa81d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa840b46e0 0x7faa8414dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa81d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa840b46e0 0x7faa8414dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35552/0 (socket says 172.21.15.163:35552) 2024-05-08T06:36:46.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa81d9b640 1 -- 172.21.15.163:0/3312856239 learned_addr learned my addr 172.21.15.163:0/3312856239 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:46.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa8259c640 1 --2- 172.21.15.163:0/3312856239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa840ae180 0x7faa8414d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa82d9d640 1 --2- 172.21.15.163:0/3312856239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa8415c020 0x7faa8414e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.131 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa81d9b640 1 -- 172.21.15.163:0/3312856239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa8415c020 msgr2=0x7faa8414e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.131 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa81d9b640 1 --2- 172.21.15.163:0/3312856239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa8415c020 0x7faa8414e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.131 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa81d9b640 1 -- 172.21.15.163:0/3312856239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa840ae180 msgr2=0x7faa8414d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.131 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa81d9b640 1 --2- 172.21.15.163:0/3312856239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa840ae180 0x7faa8414d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.131 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.129+0000 7faa81d9b640 1 -- 172.21.15.163:0/3312856239 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8414d7a0 con 0x7faa840b46e0 2024-05-08T06:36:46.131 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.130+0000 7faa81d9b640 1 --2- 172.21.15.163:0/3312856239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa840b46e0 0x7faa8414dce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faa78000c00 tx=0x7faa780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:46.131 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.130+0000 7faa72ffd640 1 -- 172.21.15.163:0/3312856239 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78041070 con 0x7faa840b46e0 2024-05-08T06:36:46.131 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.130+0000 7faa72ffd640 1 -- 172.21.15.163:0/3312856239 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa7803c670 con 0x7faa840b46e0 2024-05-08T06:36:46.131 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.130+0000 7faa88bac640 1 -- 172.21.15.163:0/3312856239 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa84151cf0 con 0x7faa840b46e0 2024-05-08T06:36:46.131 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.130+0000 7faa72ffd640 1 -- 172.21.15.163:0/3312856239 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78037040 con 0x7faa840b46e0 2024-05-08T06:36:46.131 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.130+0000 7faa88bac640 1 -- 172.21.15.163:0/3312856239 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa84151730 con 0x7faa840b46e0 2024-05-08T06:36:46.131 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.131+0000 7faa88bac640 1 -- 172.21.15.163:0/3312856239 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa8406a550 con 0x7faa840b46e0 2024-05-08T06:36:46.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.131+0000 7faa72ffd640 1 -- 172.21.15.163:0/3312856239 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faa78044070 con 0x7faa840b46e0 2024-05-08T06:36:46.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.132+0000 7faa72ffd640 1 --2- 172.21.15.163:0/3312856239 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7faa640762d0 0x7faa64078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.132+0000 7faa8259c640 1 --2- 172.21.15.163:0/3312856239 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7faa640762d0 0x7faa64078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.132+0000 7faa72ffd640 1 -- 172.21.15.163:0/3312856239 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faa780b7d10 con 0x7faa840b46e0 2024-05-08T06:36:46.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.133+0000 7faa8259c640 1 --2- 172.21.15.163:0/3312856239 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7faa640762d0 0x7faa64078790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa74000f60 tx=0x7faa74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:46.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.136+0000 7faa72ffd640 1 -- 172.21.15.163:0/3312856239 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faa78081670 con 0x7faa840b46e0 2024-05-08T06:36:46.312 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.312+0000 7faa88bac640 1 -- 172.21.15.163:0/3312856239 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa8406d5a0 con 0x7faa640762d0 2024-05-08T06:36:46.317 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.318+0000 7faa72ffd640 1 -- 172.21.15.163:0/3312856239 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7faa8406d5a0 con 0x7faa640762d0 2024-05-08T06:36:46.320 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:46.321 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2024-05-08T06:36:46.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.321+0000 7faa88bac640 1 -- 172.21.15.163:0/3312856239 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7faa640762d0 msgr2=0x7faa64078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.321+0000 7faa88bac640 1 --2- 172.21.15.163:0/3312856239 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7faa640762d0 0x7faa64078790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa74000f60 tx=0x7faa74009040 comp rx=0 tx=0).stop 2024-05-08T06:36:46.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.321+0000 7faa88bac640 1 -- 172.21.15.163:0/3312856239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa840b46e0 msgr2=0x7faa8414dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.321+0000 7faa88bac640 1 --2- 172.21.15.163:0/3312856239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa840b46e0 0x7faa8414dce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faa78000c00 tx=0x7faa780046c0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.321+0000 7faa88bac640 1 -- 172.21.15.163:0/3312856239 shutdown_connections 2024-05-08T06:36:46.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.321+0000 7faa88bac640 1 --2- 172.21.15.163:0/3312856239 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7faa640762d0 0x7faa64078790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.321+0000 7faa88bac640 1 --2- 172.21.15.163:0/3312856239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa8415c020 0x7faa8414e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.321+0000 7faa88bac640 1 --2- 172.21.15.163:0/3312856239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa840b46e0 0x7faa8414dce0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.321+0000 7faa88bac640 1 --2- 172.21.15.163:0/3312856239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa840ae180 0x7faa8414d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.321+0000 7faa88bac640 1 -- 172.21.15.163:0/3312856239 >> 172.21.15.163:0/3312856239 conn(0x7faa8406c500 msgr2=0x7faa840afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:46.324 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.322+0000 7faa88bac640 1 -- 172.21.15.163:0/3312856239 shutdown_connections 2024-05-08T06:36:46.324 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.322+0000 7faa88bac640 1 -- 172.21.15.163:0/3312856239 wait complete. 2024-05-08T06:36:46.332 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-05-08T06:36:46.098319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001183"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109935+0000","last_change":"2024-05-08T06:36:41.664385+0000","last_active":"2024-05-08T06:36:43.109935+0000","last_peered":"2024-05-08T06:36:43.109935+0000","last_clean":"2024-05-08T06:36:43.109935+0000","last_became_active":"2024-05-08T06:36:41.664106+0000","last_became_peered":"2024-05-08T06:36:41.664106+0000","last_unstale":"2024-05-08T06:36:43.109935+0000","last_undegraded":"2024-05-08T06:36:43.109935+0000","last_fullsized":"2024-05-08T06:36:43.109935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:08.774451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649403+0000","last_change":"2024-05-08T06:36:41.647763+0000","last_active":"2024-05-08T06:36:42.649403+0000","last_peered":"2024-05-08T06:36:42.649403+0000","last_clean":"2024-05-08T06:36:42.649403+0000","last_became_active":"2024-05-08T06:36:41.647487+0000","last_became_peered":"2024-05-08T06:36:41.647487+0000","last_unstale":"2024-05-08T06:36:42.649403+0000","last_undegraded":"2024-05-08T06:36:42.649403+0000","last_fullsized":"2024-05-08T06:36:42.649403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108764+0000","last_change":"2024-05-08T06:36:41.649945+0000","last_active":"2024-05-08T06:36:43.108764+0000","last_peered":"2024-05-08T06:36:43.108764+0000","last_clean":"2024-05-08T06:36:43.108764+0000","last_became_active":"2024-05-08T06:36:41.649665+0000","last_became_peered":"2024-05-08T06:36:41.649665+0000","last_unstale":"2024-05-08T06:36:43.108764+0000","last_undegraded":"2024-05-08T06:36:43.108764+0000","last_fullsized":"2024-05-08T06:36:43.108764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109325+0000","last_change":"2024-05-08T06:36:41.656179+0000","last_active":"2024-05-08T06:36:43.109325+0000","last_peered":"2024-05-08T06:36:43.109325+0000","last_clean":"2024-05-08T06:36:43.109325+0000","last_became_active":"2024-05-08T06:36:41.655649+0000","last_became_peered":"2024-05-08T06:36:41.655649+0000","last_unstale":"2024-05-08T06:36:43.109325+0000","last_undegraded":"2024-05-08T06:36:43.109325+0000","last_fullsized":"2024-05-08T06:36:43.109325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:45:07.961465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109228+0000","last_change":"2024-05-08T06:36:41.659928+0000","last_active":"2024-05-08T06:36:43.109228+0000","last_peered":"2024-05-08T06:36:43.109228+0000","last_clean":"2024-05-08T06:36:43.109228+0000","last_became_active":"2024-05-08T06:36:41.659408+0000","last_became_peered":"2024-05-08T06:36:41.659408+0000","last_unstale":"2024-05-08T06:36:43.109228+0000","last_undegraded":"2024-05-08T06:36:43.109228+0000","last_fullsized":"2024-05-08T06:36:43.109228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:14.886723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109064+0000","last_change":"2024-05-08T06:36:41.653658+0000","last_active":"2024-05-08T06:36:43.109064+0000","last_peered":"2024-05-08T06:36:43.109064+0000","last_clean":"2024-05-08T06:36:43.109064+0000","last_became_active":"2024-05-08T06:36:41.653285+0000","last_became_peered":"2024-05-08T06:36:41.653285+0000","last_unstale":"2024-05-08T06:36:43.109064+0000","last_undegraded":"2024-05-08T06:36:43.109064+0000","last_fullsized":"2024-05-08T06:36:43.109064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108884+0000","last_change":"2024-05-08T06:36:41.661193+0000","last_active":"2024-05-08T06:36:43.108884+0000","last_peered":"2024-05-08T06:36:43.108884+0000","last_clean":"2024-05-08T06:36:43.108884+0000","last_became_active":"2024-05-08T06:36:41.660907+0000","last_became_peered":"2024-05-08T06:36:41.660907+0000","last_unstale":"2024-05-08T06:36:43.108884+0000","last_undegraded":"2024-05-08T06:36:43.108884+0000","last_fullsized":"2024-05-08T06:36:43.108884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108493+0000","last_change":"2024-05-08T06:36:41.655350+0000","last_active":"2024-05-08T06:36:43.108493+0000","last_peered":"2024-05-08T06:36:43.108493+0000","last_clean":"2024-05-08T06:36:43.108493+0000","last_became_active":"2024-05-08T06:36:41.655064+0000","last_became_peered":"2024-05-08T06:36:41.655064+0000","last_unstale":"2024-05-08T06:36:43.108493+0000","last_undegraded":"2024-05-08T06:36:43.108493+0000","last_fullsized":"2024-05-08T06:36:43.108493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:11:54.263628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109972+0000","last_change":"2024-05-08T06:36:41.663210+0000","last_active":"2024-05-08T06:36:43.109972+0000","last_peered":"2024-05-08T06:36:43.109972+0000","last_clean":"2024-05-08T06:36:43.109972+0000","last_became_active":"2024-05-08T06:36:41.662954+0000","last_became_peered":"2024-05-08T06:36:41.662954+0000","last_unstale":"2024-05-08T06:36:43.109972+0000","last_undegraded":"2024-05-08T06:36:43.109972+0000","last_fullsized":"2024-05-08T06:36:43.109972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:36:32.652292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654330+0000","last_change":"2024-05-08T06:36:41.662910+0000","last_active":"2024-05-08T06:36:42.654330+0000","last_peered":"2024-05-08T06:36:42.654330+0000","last_clean":"2024-05-08T06:36:42.654330+0000","last_became_active":"2024-05-08T06:36:41.662585+0000","last_became_peered":"2024-05-08T06:36:41.662585+0000","last_unstale":"2024-05-08T06:36:42.654330+0000","last_undegraded":"2024-05-08T06:36:42.654330+0000","last_fullsized":"2024-05-08T06:36:42.654330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653737+0000","last_change":"2024-05-08T06:36:41.663863+0000","last_active":"2024-05-08T06:36:42.653737+0000","last_peered":"2024-05-08T06:36:42.653737+0000","last_clean":"2024-05-08T06:36:42.653737+0000","last_became_active":"2024-05-08T06:36:41.663584+0000","last_became_peered":"2024-05-08T06:36:41.663584+0000","last_unstale":"2024-05-08T06:36:42.653737+0000","last_undegraded":"2024-05-08T06:36:42.653737+0000","last_fullsized":"2024-05-08T06:36:42.653737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654617+0000","last_change":"2024-05-08T06:36:41.661979+0000","last_active":"2024-05-08T06:36:42.654617+0000","last_peered":"2024-05-08T06:36:42.654617+0000","last_clean":"2024-05-08T06:36:42.654617+0000","last_became_active":"2024-05-08T06:36:41.660222+0000","last_became_peered":"2024-05-08T06:36:41.660222+0000","last_unstale":"2024-05-08T06:36:42.654617+0000","last_undegraded":"2024-05-08T06:36:42.654617+0000","last_fullsized":"2024-05-08T06:36:42.654617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:26.842934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108871+0000","last_change":"2024-05-08T06:36:41.654832+0000","last_active":"2024-05-08T06:36:43.108871+0000","last_peered":"2024-05-08T06:36:43.108871+0000","last_clean":"2024-05-08T06:36:43.108871+0000","last_became_active":"2024-05-08T06:36:41.654550+0000","last_became_peered":"2024-05-08T06:36:41.654550+0000","last_unstale":"2024-05-08T06:36:43.108871+0000","last_undegraded":"2024-05-08T06:36:43.108871+0000","last_fullsized":"2024-05-08T06:36:43.108871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108766+0000","last_change":"2024-05-08T06:36:41.642429+0000","last_active":"2024-05-08T06:36:43.108766+0000","last_peered":"2024-05-08T06:36:43.108766+0000","last_clean":"2024-05-08T06:36:43.108766+0000","last_became_active":"2024-05-08T06:36:41.641956+0000","last_became_peered":"2024-05-08T06:36:41.641956+0000","last_unstale":"2024-05-08T06:36:43.108766+0000","last_undegraded":"2024-05-08T06:36:43.108766+0000","last_fullsized":"2024-05-08T06:36:43.108766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649200+0000","last_change":"2024-05-08T06:36:41.656423+0000","last_active":"2024-05-08T06:36:42.649200+0000","last_peered":"2024-05-08T06:36:42.649200+0000","last_clean":"2024-05-08T06:36:42.649200+0000","last_became_active":"2024-05-08T06:36:41.656154+0000","last_became_peered":"2024-05-08T06:36:41.656154+0000","last_unstale":"2024-05-08T06:36:42.649200+0000","last_undegraded":"2024-05-08T06:36:42.649200+0000","last_fullsized":"2024-05-08T06:36:42.649200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109681+0000","last_change":"2024-05-08T06:36:41.653378+0000","last_active":"2024-05-08T06:36:43.109681+0000","last_peered":"2024-05-08T06:36:43.109681+0000","last_clean":"2024-05-08T06:36:43.109681+0000","last_became_active":"2024-05-08T06:36:41.653027+0000","last_became_peered":"2024-05-08T06:36:41.653027+0000","last_unstale":"2024-05-08T06:36:43.109681+0000","last_undegraded":"2024-05-08T06:36:43.109681+0000","last_fullsized":"2024-05-08T06:36:43.109681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:02.654155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109919+0000","last_change":"2024-05-08T06:36:41.659055+0000","last_active":"2024-05-08T06:36:43.109919+0000","last_peered":"2024-05-08T06:36:43.109919+0000","last_clean":"2024-05-08T06:36:43.109919+0000","last_became_active":"2024-05-08T06:36:41.658774+0000","last_became_peered":"2024-05-08T06:36:41.658774+0000","last_unstale":"2024-05-08T06:36:43.109919+0000","last_undegraded":"2024-05-08T06:36:43.109919+0000","last_fullsized":"2024-05-08T06:36:43.109919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:38:12.052561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110931+0000","last_change":"2024-05-08T06:36:41.649669+0000","last_active":"2024-05-08T06:36:43.110931+0000","last_peered":"2024-05-08T06:36:43.110931+0000","last_clean":"2024-05-08T06:36:43.110931+0000","last_became_active":"2024-05-08T06:36:41.649391+0000","last_became_peered":"2024-05-08T06:36:41.649391+0000","last_unstale":"2024-05-08T06:36:43.110931+0000","last_undegraded":"2024-05-08T06:36:43.110931+0000","last_fullsized":"2024-05-08T06:36:43.110931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:04.579668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109443+0000","last_change":"2024-05-08T06:36:41.660852+0000","last_active":"2024-05-08T06:36:43.109443+0000","last_peered":"2024-05-08T06:36:43.109443+0000","last_clean":"2024-05-08T06:36:43.109443+0000","last_became_active":"2024-05-08T06:36:41.660573+0000","last_became_peered":"2024-05-08T06:36:41.660573+0000","last_unstale":"2024-05-08T06:36:43.109443+0000","last_undegraded":"2024-05-08T06:36:43.109443+0000","last_fullsized":"2024-05-08T06:36:43.109443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:02.654155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653881+0000","last_change":"2024-05-08T06:36:41.663211+0000","last_active":"2024-05-08T06:36:42.653881+0000","last_peered":"2024-05-08T06:36:42.653881+0000","last_clean":"2024-05-08T06:36:42.653881+0000","last_became_active":"2024-05-08T06:36:41.662936+0000","last_became_peered":"2024-05-08T06:36:41.662936+0000","last_unstale":"2024-05-08T06:36:42.653881+0000","last_undegraded":"2024-05-08T06:36:42.653881+0000","last_fullsized":"2024-05-08T06:36:42.653881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108430+0000","last_change":"2024-05-08T06:36:41.661453+0000","last_active":"2024-05-08T06:36:43.108430+0000","last_peered":"2024-05-08T06:36:43.108430+0000","last_clean":"2024-05-08T06:36:43.108430+0000","last_became_active":"2024-05-08T06:36:41.661146+0000","last_became_peered":"2024-05-08T06:36:41.661146+0000","last_unstale":"2024-05-08T06:36:43.108430+0000","last_undegraded":"2024-05-08T06:36:43.108430+0000","last_fullsized":"2024-05-08T06:36:43.108430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:11:54.263628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109377+0000","last_change":"2024-05-08T06:36:41.658664+0000","last_active":"2024-05-08T06:36:43.109377+0000","last_peered":"2024-05-08T06:36:43.109377+0000","last_clean":"2024-05-08T06:36:43.109377+0000","last_became_active":"2024-05-08T06:36:41.658352+0000","last_became_peered":"2024-05-08T06:36:41.658352+0000","last_unstale":"2024-05-08T06:36:43.109377+0000","last_undegraded":"2024-05-08T06:36:43.109377+0000","last_fullsized":"2024-05-08T06:36:43.109377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108694+0000","last_change":"2024-05-08T06:36:41.658967+0000","last_active":"2024-05-08T06:36:43.108694+0000","last_peered":"2024-05-08T06:36:43.108694+0000","last_clean":"2024-05-08T06:36:43.108694+0000","last_became_active":"2024-05-08T06:36:41.658387+0000","last_became_peered":"2024-05-08T06:36:41.658387+0000","last_unstale":"2024-05-08T06:36:43.108694+0000","last_undegraded":"2024-05-08T06:36:43.108694+0000","last_fullsized":"2024-05-08T06:36:43.108694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108892+0000","last_change":"2024-05-08T06:36:41.657102+0000","last_active":"2024-05-08T06:36:43.108892+0000","last_peered":"2024-05-08T06:36:43.108892+0000","last_clean":"2024-05-08T06:36:43.108892+0000","last_became_active":"2024-05-08T06:36:41.656844+0000","last_became_peered":"2024-05-08T06:36:41.656844+0000","last_unstale":"2024-05-08T06:36:43.108892+0000","last_undegraded":"2024-05-08T06:36:43.108892+0000","last_fullsized":"2024-05-08T06:36:43.108892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109150+0000","last_change":"2024-05-08T06:36:41.659592+0000","last_active":"2024-05-08T06:36:43.109150+0000","last_peered":"2024-05-08T06:36:43.109150+0000","last_clean":"2024-05-08T06:36:43.109150+0000","last_became_active":"2024-05-08T06:36:41.659295+0000","last_became_peered":"2024-05-08T06:36:41.659295+0000","last_unstale":"2024-05-08T06:36:43.109150+0000","last_undegraded":"2024-05-08T06:36:43.109150+0000","last_fullsized":"2024-05-08T06:36:43.109150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654479+0000","last_change":"2024-05-08T06:36:41.649572+0000","last_active":"2024-05-08T06:36:42.654479+0000","last_peered":"2024-05-08T06:36:42.654479+0000","last_clean":"2024-05-08T06:36:42.654479+0000","last_became_active":"2024-05-08T06:36:41.649302+0000","last_became_peered":"2024-05-08T06:36:41.649302+0000","last_unstale":"2024-05-08T06:36:42.654479+0000","last_undegraded":"2024-05-08T06:36:42.654479+0000","last_fullsized":"2024-05-08T06:36:42.654479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649400+0000","last_change":"2024-05-08T06:36:41.657182+0000","last_active":"2024-05-08T06:36:42.649400+0000","last_peered":"2024-05-08T06:36:42.649400+0000","last_clean":"2024-05-08T06:36:42.649400+0000","last_became_active":"2024-05-08T06:36:41.656919+0000","last_became_peered":"2024-05-08T06:36:41.656919+0000","last_unstale":"2024-05-08T06:36:42.649400+0000","last_undegraded":"2024-05-08T06:36:42.649400+0000","last_fullsized":"2024-05-08T06:36:42.649400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109121+0000","last_change":"2024-05-08T06:36:41.659147+0000","last_active":"2024-05-08T06:36:43.109121+0000","last_peered":"2024-05-08T06:36:43.109121+0000","last_clean":"2024-05-08T06:36:43.109121+0000","last_became_active":"2024-05-08T06:36:41.658880+0000","last_became_peered":"2024-05-08T06:36:41.658880+0000","last_unstale":"2024-05-08T06:36:43.109121+0000","last_undegraded":"2024-05-08T06:36:43.109121+0000","last_fullsized":"2024-05-08T06:36:43.109121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108909+0000","last_change":"2024-05-08T06:36:41.658925+0000","last_active":"2024-05-08T06:36:43.108909+0000","last_peered":"2024-05-08T06:36:43.108909+0000","last_clean":"2024-05-08T06:36:43.108909+0000","last_became_active":"2024-05-08T06:36:41.656879+0000","last_became_peered":"2024-05-08T06:36:41.656879+0000","last_unstale":"2024-05-08T06:36:43.108909+0000","last_undegraded":"2024-05-08T06:36:43.108909+0000","last_fullsized":"2024-05-08T06:36:43.108909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109551+0000","last_change":"2024-05-08T06:36:41.655125+0000","last_active":"2024-05-08T06:36:43.109551+0000","last_peered":"2024-05-08T06:36:43.109551+0000","last_clean":"2024-05-08T06:36:43.109551+0000","last_became_active":"2024-05-08T06:36:41.654817+0000","last_became_peered":"2024-05-08T06:36:41.654817+0000","last_unstale":"2024-05-08T06:36:43.109551+0000","last_undegraded":"2024-05-08T06:36:43.109551+0000","last_fullsized":"2024-05-08T06:36:43.109551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:42.444444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110786+0000","last_change":"2024-05-08T06:36:41.655361+0000","last_active":"2024-05-08T06:36:43.110786+0000","last_peered":"2024-05-08T06:36:43.110786+0000","last_clean":"2024-05-08T06:36:43.110786+0000","last_became_active":"2024-05-08T06:36:41.655085+0000","last_became_peered":"2024-05-08T06:36:41.655085+0000","last_unstale":"2024-05-08T06:36:43.110786+0000","last_undegraded":"2024-05-08T06:36:43.110786+0000","last_fullsized":"2024-05-08T06:36:43.110786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:26.842934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653569+0000","last_change":"2024-05-08T06:36:41.643791+0000","last_active":"2024-05-08T06:36:42.653569+0000","last_peered":"2024-05-08T06:36:42.653569+0000","last_clean":"2024-05-08T06:36:42.653569+0000","last_became_active":"2024-05-08T06:36:41.643506+0000","last_became_peered":"2024-05-08T06:36:41.643506+0000","last_unstale":"2024-05-08T06:36:42.653569+0000","last_undegraded":"2024-05-08T06:36:42.653569+0000","last_fullsized":"2024-05-08T06:36:42.653569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654375+0000","last_change":"2024-05-08T06:36:41.654106+0000","last_active":"2024-05-08T06:36:42.654375+0000","last_peered":"2024-05-08T06:36:42.654375+0000","last_clean":"2024-05-08T06:36:42.654375+0000","last_became_active":"2024-05-08T06:36:41.653822+0000","last_became_peered":"2024-05-08T06:36:41.653822+0000","last_unstale":"2024-05-08T06:36:42.654375+0000","last_undegraded":"2024-05-08T06:36:42.654375+0000","last_fullsized":"2024-05-08T06:36:42.654375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:51.358786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109209+0000","last_change":"2024-05-08T06:36:40.641349+0000","last_active":"2024-05-08T06:36:43.109209+0000","last_peered":"2024-05-08T06:36:43.109209+0000","last_clean":"2024-05-08T06:36:43.109209+0000","last_became_active":"2024-05-08T06:36:40.641086+0000","last_became_peered":"2024-05-08T06:36:40.641086+0000","last_unstale":"2024-05-08T06:36:43.109209+0000","last_undegraded":"2024-05-08T06:36:43.109209+0000","last_fullsized":"2024-05-08T06:36:43.109209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:39:09.616149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108579+0000","last_change":"2024-05-08T06:36:41.656120+0000","last_active":"2024-05-08T06:36:43.108579+0000","last_peered":"2024-05-08T06:36:43.108579+0000","last_clean":"2024-05-08T06:36:43.108579+0000","last_became_active":"2024-05-08T06:36:41.655843+0000","last_became_peered":"2024-05-08T06:36:41.655843+0000","last_unstale":"2024-05-08T06:36:43.108579+0000","last_undegraded":"2024-05-08T06:36:43.108579+0000","last_fullsized":"2024-05-08T06:36:43.108579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654066+0000","last_change":"2024-05-08T06:36:40.639740+0000","last_active":"2024-05-08T06:36:42.654066+0000","last_peered":"2024-05-08T06:36:42.654066+0000","last_clean":"2024-05-08T06:36:42.654066+0000","last_became_active":"2024-05-08T06:36:40.639370+0000","last_became_peered":"2024-05-08T06:36:40.639370+0000","last_unstale":"2024-05-08T06:36:42.654066+0000","last_undegraded":"2024-05-08T06:36:42.654066+0000","last_fullsized":"2024-05-08T06:36:42.654066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108684+0000","last_change":"2024-05-08T06:36:40.641983+0000","last_active":"2024-05-08T06:36:43.108684+0000","last_peered":"2024-05-08T06:36:43.108684+0000","last_clean":"2024-05-08T06:36:43.108684+0000","last_became_active":"2024-05-08T06:36:40.641666+0000","last_became_peered":"2024-05-08T06:36:40.641666+0000","last_unstale":"2024-05-08T06:36:43.108684+0000","last_undegraded":"2024-05-08T06:36:43.108684+0000","last_fullsized":"2024-05-08T06:36:43.108684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:47.997246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108516+0000","last_change":"2024-05-08T06:36:41.653257+0000","last_active":"2024-05-08T06:36:43.108516+0000","last_peered":"2024-05-08T06:36:43.108516+0000","last_clean":"2024-05-08T06:36:43.108516+0000","last_became_active":"2024-05-08T06:36:41.652900+0000","last_became_peered":"2024-05-08T06:36:41.652900+0000","last_unstale":"2024-05-08T06:36:43.108516+0000","last_undegraded":"2024-05-08T06:36:43.108516+0000","last_fullsized":"2024-05-08T06:36:43.108516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109272+0000","last_change":"2024-05-08T06:36:40.646856+0000","last_active":"2024-05-08T06:36:43.109272+0000","last_peered":"2024-05-08T06:36:43.109272+0000","last_clean":"2024-05-08T06:36:43.109272+0000","last_became_active":"2024-05-08T06:36:40.644640+0000","last_became_peered":"2024-05-08T06:36:40.644640+0000","last_unstale":"2024-05-08T06:36:43.109272+0000","last_undegraded":"2024-05-08T06:36:43.109272+0000","last_fullsized":"2024-05-08T06:36:43.109272+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649183+0000","last_change":"2024-05-08T06:36:41.656744+0000","last_active":"2024-05-08T06:36:42.649183+0000","last_peered":"2024-05-08T06:36:42.649183+0000","last_clean":"2024-05-08T06:36:42.649183+0000","last_became_active":"2024-05-08T06:36:41.656483+0000","last_became_peered":"2024-05-08T06:36:41.656483+0000","last_unstale":"2024-05-08T06:36:42.649183+0000","last_undegraded":"2024-05-08T06:36:42.649183+0000","last_fullsized":"2024-05-08T06:36:42.649183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653768+0000","last_change":"2024-05-08T06:36:40.648551+0000","last_active":"2024-05-08T06:36:42.653768+0000","last_peered":"2024-05-08T06:36:42.653768+0000","last_clean":"2024-05-08T06:36:42.653768+0000","last_became_active":"2024-05-08T06:36:40.648265+0000","last_became_peered":"2024-05-08T06:36:40.648265+0000","last_unstale":"2024-05-08T06:36:42.653768+0000","last_undegraded":"2024-05-08T06:36:42.653768+0000","last_fullsized":"2024-05-08T06:36:42.653768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109024+0000","last_change":"2024-05-08T06:36:41.658637+0000","last_active":"2024-05-08T06:36:43.109024+0000","last_peered":"2024-05-08T06:36:43.109024+0000","last_clean":"2024-05-08T06:36:43.109024+0000","last_became_active":"2024-05-08T06:36:41.658333+0000","last_became_peered":"2024-05-08T06:36:41.658333+0000","last_unstale":"2024-05-08T06:36:43.109024+0000","last_undegraded":"2024-05-08T06:36:43.109024+0000","last_fullsized":"2024-05-08T06:36:43.109024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109045+0000","last_change":"2024-05-08T06:36:40.647645+0000","last_active":"2024-05-08T06:36:43.109045+0000","last_peered":"2024-05-08T06:36:43.109045+0000","last_clean":"2024-05-08T06:36:43.109045+0000","last_became_active":"2024-05-08T06:36:40.647368+0000","last_became_peered":"2024-05-08T06:36:40.647368+0000","last_unstale":"2024-05-08T06:36:43.109045+0000","last_undegraded":"2024-05-08T06:36:43.109045+0000","last_fullsized":"2024-05-08T06:36:43.109045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109122+0000","last_change":"2024-05-08T06:36:41.662012+0000","last_active":"2024-05-08T06:36:43.109122+0000","last_peered":"2024-05-08T06:36:43.109122+0000","last_clean":"2024-05-08T06:36:43.109122+0000","last_became_active":"2024-05-08T06:36:41.661780+0000","last_became_peered":"2024-05-08T06:36:41.661780+0000","last_unstale":"2024-05-08T06:36:43.109122+0000","last_undegraded":"2024-05-08T06:36:43.109122+0000","last_fullsized":"2024-05-08T06:36:43.109122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649372+0000","last_change":"2024-05-08T06:36:40.649405+0000","last_active":"2024-05-08T06:36:42.649372+0000","last_peered":"2024-05-08T06:36:42.649372+0000","last_clean":"2024-05-08T06:36:42.649372+0000","last_became_active":"2024-05-08T06:36:40.649172+0000","last_became_peered":"2024-05-08T06:36:40.649172+0000","last_unstale":"2024-05-08T06:36:42.649372+0000","last_undegraded":"2024-05-08T06:36:42.649372+0000","last_fullsized":"2024-05-08T06:36:42.649372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654679+0000","last_change":"2024-05-08T06:36:41.654141+0000","last_active":"2024-05-08T06:36:42.654679+0000","last_peered":"2024-05-08T06:36:42.654679+0000","last_clean":"2024-05-08T06:36:42.654679+0000","last_became_active":"2024-05-08T06:36:41.653723+0000","last_became_peered":"2024-05-08T06:36:41.653723+0000","last_unstale":"2024-05-08T06:36:42.654679+0000","last_undegraded":"2024-05-08T06:36:42.654679+0000","last_fullsized":"2024-05-08T06:36:42.654679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:04.579668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654000+0000","last_change":"2024-05-08T06:36:40.641783+0000","last_active":"2024-05-08T06:36:42.654000+0000","last_peered":"2024-05-08T06:36:42.654000+0000","last_clean":"2024-05-08T06:36:42.654000+0000","last_became_active":"2024-05-08T06:36:40.641270+0000","last_became_peered":"2024-05-08T06:36:40.641270+0000","last_unstale":"2024-05-08T06:36:42.654000+0000","last_undegraded":"2024-05-08T06:36:42.654000+0000","last_fullsized":"2024-05-08T06:36:42.654000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108979+0000","last_change":"2024-05-08T06:36:41.657579+0000","last_active":"2024-05-08T06:36:43.108979+0000","last_peered":"2024-05-08T06:36:43.108979+0000","last_clean":"2024-05-08T06:36:43.108979+0000","last_became_active":"2024-05-08T06:36:41.657299+0000","last_became_peered":"2024-05-08T06:36:41.657299+0000","last_unstale":"2024-05-08T06:36:43.108979+0000","last_undegraded":"2024-05-08T06:36:43.108979+0000","last_fullsized":"2024-05-08T06:36:43.108979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:02.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108685+0000","last_change":"2024-05-08T06:36:40.641651+0000","last_active":"2024-05-08T06:36:43.108685+0000","last_peered":"2024-05-08T06:36:43.108685+0000","last_clean":"2024-05-08T06:36:43.108685+0000","last_became_active":"2024-05-08T06:36:40.641168+0000","last_became_peered":"2024-05-08T06:36:40.641168+0000","last_unstale":"2024-05-08T06:36:43.108685+0000","last_undegraded":"2024-05-08T06:36:43.108685+0000","last_fullsized":"2024-05-08T06:36:43.108685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:47.997246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108903+0000","last_change":"2024-05-08T06:36:41.661356+0000","last_active":"2024-05-08T06:36:43.108903+0000","last_peered":"2024-05-08T06:36:43.108903+0000","last_clean":"2024-05-08T06:36:43.108903+0000","last_became_active":"2024-05-08T06:36:41.661085+0000","last_became_peered":"2024-05-08T06:36:41.661085+0000","last_unstale":"2024-05-08T06:36:43.108903+0000","last_undegraded":"2024-05-08T06:36:43.108903+0000","last_fullsized":"2024-05-08T06:36:43.108903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109770+0000","last_change":"2024-05-08T06:36:40.646694+0000","last_active":"2024-05-08T06:36:43.109770+0000","last_peered":"2024-05-08T06:36:43.109770+0000","last_clean":"2024-05-08T06:36:43.109770+0000","last_became_active":"2024-05-08T06:36:40.645833+0000","last_became_peered":"2024-05-08T06:36:40.645833+0000","last_unstale":"2024-05-08T06:36:43.109770+0000","last_undegraded":"2024-05-08T06:36:43.109770+0000","last_fullsized":"2024-05-08T06:36:43.109770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109418+0000","last_change":"2024-05-08T06:36:41.668787+0000","last_active":"2024-05-08T06:36:43.109418+0000","last_peered":"2024-05-08T06:36:43.109418+0000","last_clean":"2024-05-08T06:36:43.109418+0000","last_became_active":"2024-05-08T06:36:41.668489+0000","last_became_peered":"2024-05-08T06:36:41.668489+0000","last_unstale":"2024-05-08T06:36:43.109418+0000","last_undegraded":"2024-05-08T06:36:43.109418+0000","last_fullsized":"2024-05-08T06:36:43.109418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.395237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653600+0000","last_change":"2024-05-08T06:36:40.640082+0000","last_active":"2024-05-08T06:36:42.653600+0000","last_peered":"2024-05-08T06:36:42.653600+0000","last_clean":"2024-05-08T06:36:42.653600+0000","last_became_active":"2024-05-08T06:36:40.638558+0000","last_became_peered":"2024-05-08T06:36:40.638558+0000","last_unstale":"2024-05-08T06:36:42.653600+0000","last_undegraded":"2024-05-08T06:36:42.653600+0000","last_fullsized":"2024-05-08T06:36:42.653600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653523+0000","last_change":"2024-05-08T06:36:41.643855+0000","last_active":"2024-05-08T06:36:42.653523+0000","last_peered":"2024-05-08T06:36:42.653523+0000","last_clean":"2024-05-08T06:36:42.653523+0000","last_became_active":"2024-05-08T06:36:41.643117+0000","last_became_peered":"2024-05-08T06:36:41.643117+0000","last_unstale":"2024-05-08T06:36:42.653523+0000","last_undegraded":"2024-05-08T06:36:42.653523+0000","last_fullsized":"2024-05-08T06:36:42.653523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110713+0000","last_change":"2024-05-08T06:36:40.642303+0000","last_active":"2024-05-08T06:36:43.110713+0000","last_peered":"2024-05-08T06:36:43.110713+0000","last_clean":"2024-05-08T06:36:43.110713+0000","last_became_active":"2024-05-08T06:36:40.642042+0000","last_became_peered":"2024-05-08T06:36:40.642042+0000","last_unstale":"2024-05-08T06:36:43.110713+0000","last_undegraded":"2024-05-08T06:36:43.110713+0000","last_fullsized":"2024-05-08T06:36:43.110713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109058+0000","last_change":"2024-05-08T06:36:41.660559+0000","last_active":"2024-05-08T06:36:43.109058+0000","last_peered":"2024-05-08T06:36:43.109058+0000","last_clean":"2024-05-08T06:36:43.109058+0000","last_became_active":"2024-05-08T06:36:41.660282+0000","last_became_peered":"2024-05-08T06:36:41.660282+0000","last_unstale":"2024-05-08T06:36:43.109058+0000","last_undegraded":"2024-05-08T06:36:43.109058+0000","last_fullsized":"2024-05-08T06:36:43.109058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:49.740862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654374+0000","last_change":"2024-05-08T06:36:40.646655+0000","last_active":"2024-05-08T06:36:42.654374+0000","last_peered":"2024-05-08T06:36:42.654374+0000","last_clean":"2024-05-08T06:36:42.654374+0000","last_became_active":"2024-05-08T06:36:40.646346+0000","last_became_peered":"2024-05-08T06:36:40.646346+0000","last_unstale":"2024-05-08T06:36:42.654374+0000","last_undegraded":"2024-05-08T06:36:42.654374+0000","last_fullsized":"2024-05-08T06:36:42.654374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109890+0000","last_change":"2024-05-08T06:36:41.659817+0000","last_active":"2024-05-08T06:36:43.109890+0000","last_peered":"2024-05-08T06:36:43.109890+0000","last_clean":"2024-05-08T06:36:43.109890+0000","last_became_active":"2024-05-08T06:36:41.659524+0000","last_became_peered":"2024-05-08T06:36:41.659524+0000","last_unstale":"2024-05-08T06:36:43.109890+0000","last_undegraded":"2024-05-08T06:36:43.109890+0000","last_fullsized":"2024-05-08T06:36:43.109890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:38:12.052561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110881+0000","last_change":"2024-05-08T06:36:40.641491+0000","last_active":"2024-05-08T06:36:43.110881+0000","last_peered":"2024-05-08T06:36:43.110881+0000","last_clean":"2024-05-08T06:36:43.110881+0000","last_became_active":"2024-05-08T06:36:40.641207+0000","last_became_peered":"2024-05-08T06:36:40.641207+0000","last_unstale":"2024-05-08T06:36:43.110881+0000","last_undegraded":"2024-05-08T06:36:43.110881+0000","last_fullsized":"2024-05-08T06:36:43.110881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110049+0000","last_change":"2024-05-08T06:36:41.667992+0000","last_active":"2024-05-08T06:36:43.110049+0000","last_peered":"2024-05-08T06:36:43.110049+0000","last_clean":"2024-05-08T06:36:43.110049+0000","last_became_active":"2024-05-08T06:36:41.667733+0000","last_became_peered":"2024-05-08T06:36:41.667733+0000","last_unstale":"2024-05-08T06:36:43.110049+0000","last_undegraded":"2024-05-08T06:36:43.110049+0000","last_fullsized":"2024-05-08T06:36:43.110049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:41:00.520597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649448+0000","last_change":"2024-05-08T06:36:40.646069+0000","last_active":"2024-05-08T06:36:42.649448+0000","last_peered":"2024-05-08T06:36:42.649448+0000","last_clean":"2024-05-08T06:36:42.649448+0000","last_became_active":"2024-05-08T06:36:40.645769+0000","last_became_peered":"2024-05-08T06:36:40.645769+0000","last_unstale":"2024-05-08T06:36:42.649448+0000","last_undegraded":"2024-05-08T06:36:42.649448+0000","last_fullsized":"2024-05-08T06:36:42.649448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:25.837342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109628+0000","last_change":"2024-05-08T06:36:41.660346+0000","last_active":"2024-05-08T06:36:43.109628+0000","last_peered":"2024-05-08T06:36:43.109628+0000","last_clean":"2024-05-08T06:36:43.109628+0000","last_became_active":"2024-05-08T06:36:41.660028+0000","last_became_peered":"2024-05-08T06:36:41.660028+0000","last_unstale":"2024-05-08T06:36:43.109628+0000","last_undegraded":"2024-05-08T06:36:43.109628+0000","last_fullsized":"2024-05-08T06:36:43.109628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:14.886723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110847+0000","last_change":"2024-05-08T06:36:40.647112+0000","last_active":"2024-05-08T06:36:43.110847+0000","last_peered":"2024-05-08T06:36:43.110847+0000","last_clean":"2024-05-08T06:36:43.110847+0000","last_became_active":"2024-05-08T06:36:40.646804+0000","last_became_peered":"2024-05-08T06:36:40.646804+0000","last_unstale":"2024-05-08T06:36:43.110847+0000","last_undegraded":"2024-05-08T06:36:43.110847+0000","last_fullsized":"2024-05-08T06:36:43.110847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109148+0000","last_change":"2024-05-08T06:36:41.663875+0000","last_active":"2024-05-08T06:36:43.109148+0000","last_peered":"2024-05-08T06:36:43.109148+0000","last_clean":"2024-05-08T06:36:43.109148+0000","last_became_active":"2024-05-08T06:36:41.663595+0000","last_became_peered":"2024-05-08T06:36:41.663595+0000","last_unstale":"2024-05-08T06:36:43.109148+0000","last_undegraded":"2024-05-08T06:36:43.109148+0000","last_fullsized":"2024-05-08T06:36:43.109148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:42.444444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108725+0000","last_change":"2024-05-08T06:36:31.710287+0000","last_active":"2024-05-08T06:36:43.108725+0000","last_peered":"2024-05-08T06:36:43.108725+0000","last_clean":"2024-05-08T06:36:43.108725+0000","last_became_active":"2024-05-08T06:36:31.709270+0000","last_became_peered":"2024-05-08T06:36:31.709270+0000","last_unstale":"2024-05-08T06:36:43.108725+0000","last_undegraded":"2024-05-08T06:36:43.108725+0000","last_fullsized":"2024-05-08T06:36:43.108725+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_clean_scrub_stamp":"2024-05-08T06:36:30.122918+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:42.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108783+0000","last_change":"2024-05-08T06:36:40.645536+0000","last_active":"2024-05-08T06:36:43.108783+0000","last_peered":"2024-05-08T06:36:43.108783+0000","last_clean":"2024-05-08T06:36:43.108783+0000","last_became_active":"2024-05-08T06:36:40.644256+0000","last_became_peered":"2024-05-08T06:36:40.644256+0000","last_unstale":"2024-05-08T06:36:43.108783+0000","last_undegraded":"2024-05-08T06:36:43.108783+0000","last_fullsized":"2024-05-08T06:36:43.108783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109026+0000","last_change":"2024-05-08T06:36:41.655301+0000","last_active":"2024-05-08T06:36:43.109026+0000","last_peered":"2024-05-08T06:36:43.109026+0000","last_clean":"2024-05-08T06:36:43.109026+0000","last_became_active":"2024-05-08T06:36:41.655090+0000","last_became_peered":"2024-05-08T06:36:41.655090+0000","last_unstale":"2024-05-08T06:36:43.109026+0000","last_undegraded":"2024-05-08T06:36:43.109026+0000","last_fullsized":"2024-05-08T06:36:43.109026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108653+0000","last_change":"2024-05-08T06:36:40.647913+0000","last_active":"2024-05-08T06:36:43.108653+0000","last_peered":"2024-05-08T06:36:43.108653+0000","last_clean":"2024-05-08T06:36:43.108653+0000","last_became_active":"2024-05-08T06:36:40.647638+0000","last_became_peered":"2024-05-08T06:36:40.647638+0000","last_unstale":"2024-05-08T06:36:43.108653+0000","last_undegraded":"2024-05-08T06:36:43.108653+0000","last_fullsized":"2024-05-08T06:36:43.108653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:29:44.646218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108285+0000","last_change":"2024-05-08T06:36:41.658563+0000","last_active":"2024-05-08T06:36:43.108285+0000","last_peered":"2024-05-08T06:36:43.108285+0000","last_clean":"2024-05-08T06:36:43.108285+0000","last_became_active":"2024-05-08T06:36:41.657157+0000","last_became_peered":"2024-05-08T06:36:41.657157+0000","last_unstale":"2024-05-08T06:36:43.108285+0000","last_undegraded":"2024-05-08T06:36:43.108285+0000","last_fullsized":"2024-05-08T06:36:43.108285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:30:14.798179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110812+0000","last_change":"2024-05-08T06:36:40.642187+0000","last_active":"2024-05-08T06:36:43.110812+0000","last_peered":"2024-05-08T06:36:43.110812+0000","last_clean":"2024-05-08T06:36:43.110812+0000","last_became_active":"2024-05-08T06:36:40.641906+0000","last_became_peered":"2024-05-08T06:36:40.641906+0000","last_unstale":"2024-05-08T06:36:43.110812+0000","last_undegraded":"2024-05-08T06:36:43.110812+0000","last_fullsized":"2024-05-08T06:36:43.110812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649176+0000","last_change":"2024-05-08T06:36:41.642328+0000","last_active":"2024-05-08T06:36:42.649176+0000","last_peered":"2024-05-08T06:36:42.649176+0000","last_clean":"2024-05-08T06:36:42.649176+0000","last_became_active":"2024-05-08T06:36:41.642053+0000","last_became_peered":"2024-05-08T06:36:41.642053+0000","last_unstale":"2024-05-08T06:36:42.649176+0000","last_undegraded":"2024-05-08T06:36:42.649176+0000","last_fullsized":"2024-05-08T06:36:42.649176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109887+0000","last_change":"2024-05-08T06:36:40.646469+0000","last_active":"2024-05-08T06:36:43.109887+0000","last_peered":"2024-05-08T06:36:43.109887+0000","last_clean":"2024-05-08T06:36:43.109887+0000","last_became_active":"2024-05-08T06:36:40.646207+0000","last_became_peered":"2024-05-08T06:36:40.646207+0000","last_unstale":"2024-05-08T06:36:43.109887+0000","last_undegraded":"2024-05-08T06:36:43.109887+0000","last_fullsized":"2024-05-08T06:36:43.109887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654706+0000","last_change":"2024-05-08T06:36:41.654729+0000","last_active":"2024-05-08T06:36:42.654706+0000","last_peered":"2024-05-08T06:36:42.654706+0000","last_clean":"2024-05-08T06:36:42.654706+0000","last_became_active":"2024-05-08T06:36:41.654466+0000","last_became_peered":"2024-05-08T06:36:41.654466+0000","last_unstale":"2024-05-08T06:36:42.654706+0000","last_undegraded":"2024-05-08T06:36:42.654706+0000","last_fullsized":"2024-05-08T06:36:42.654706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:44.798191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649223+0000","last_change":"2024-05-08T06:36:40.647304+0000","last_active":"2024-05-08T06:36:42.649223+0000","last_peered":"2024-05-08T06:36:42.649223+0000","last_clean":"2024-05-08T06:36:42.649223+0000","last_became_active":"2024-05-08T06:36:40.647010+0000","last_became_peered":"2024-05-08T06:36:40.647010+0000","last_unstale":"2024-05-08T06:36:42.649223+0000","last_undegraded":"2024-05-08T06:36:42.649223+0000","last_fullsized":"2024-05-08T06:36:42.649223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:50.353194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109655+0000","last_change":"2024-05-08T06:36:41.656064+0000","last_active":"2024-05-08T06:36:43.109655+0000","last_peered":"2024-05-08T06:36:43.109655+0000","last_clean":"2024-05-08T06:36:43.109655+0000","last_became_active":"2024-05-08T06:36:41.655714+0000","last_became_peered":"2024-05-08T06:36:41.655714+0000","last_unstale":"2024-05-08T06:36:43.109655+0000","last_undegraded":"2024-05-08T06:36:43.109655+0000","last_fullsized":"2024-05-08T06:36:43.109655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.395237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109737+0000","last_change":"2024-05-08T06:36:40.648662+0000","last_active":"2024-05-08T06:36:43.109737+0000","last_peered":"2024-05-08T06:36:43.109737+0000","last_clean":"2024-05-08T06:36:43.109737+0000","last_became_active":"2024-05-08T06:36:40.648367+0000","last_became_peered":"2024-05-08T06:36:40.648367+0000","last_unstale":"2024-05-08T06:36:43.109737+0000","last_undegraded":"2024-05-08T06:36:43.109737+0000","last_fullsized":"2024-05-08T06:36:43.109737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:25.837342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649254+0000","last_change":"2024-05-08T06:36:41.642330+0000","last_active":"2024-05-08T06:36:42.649254+0000","last_peered":"2024-05-08T06:36:42.649254+0000","last_clean":"2024-05-08T06:36:42.649254+0000","last_became_active":"2024-05-08T06:36:41.642059+0000","last_became_peered":"2024-05-08T06:36:41.642059+0000","last_unstale":"2024-05-08T06:36:42.649254+0000","last_undegraded":"2024-05-08T06:36:42.649254+0000","last_fullsized":"2024-05-08T06:36:42.649254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649350+0000","last_change":"2024-05-08T06:36:40.646675+0000","last_active":"2024-05-08T06:36:42.649350+0000","last_peered":"2024-05-08T06:36:42.649350+0000","last_clean":"2024-05-08T06:36:42.649350+0000","last_became_active":"2024-05-08T06:36:40.646346+0000","last_became_peered":"2024-05-08T06:36:40.646346+0000","last_unstale":"2024-05-08T06:36:42.649350+0000","last_undegraded":"2024-05-08T06:36:42.649350+0000","last_fullsized":"2024-05-08T06:36:42.649350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:03.574076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109189+0000","last_change":"2024-05-08T06:36:41.658633+0000","last_active":"2024-05-08T06:36:43.109189+0000","last_peered":"2024-05-08T06:36:43.109189+0000","last_clean":"2024-05-08T06:36:43.109189+0000","last_became_active":"2024-05-08T06:36:41.658294+0000","last_became_peered":"2024-05-08T06:36:41.658294+0000","last_unstale":"2024-05-08T06:36:43.109189+0000","last_undegraded":"2024-05-08T06:36:43.109189+0000","last_fullsized":"2024-05-08T06:36:43.109189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653882+0000","last_change":"2024-05-08T06:36:40.639547+0000","last_active":"2024-05-08T06:36:42.653882+0000","last_peered":"2024-05-08T06:36:42.653882+0000","last_clean":"2024-05-08T06:36:42.653882+0000","last_became_active":"2024-05-08T06:36:40.639240+0000","last_became_peered":"2024-05-08T06:36:40.639240+0000","last_unstale":"2024-05-08T06:36:42.653882+0000","last_undegraded":"2024-05-08T06:36:42.653882+0000","last_fullsized":"2024-05-08T06:36:42.653882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108530+0000","last_change":"2024-05-08T06:36:41.648709+0000","last_active":"2024-05-08T06:36:43.108530+0000","last_peered":"2024-05-08T06:36:43.108530+0000","last_clean":"2024-05-08T06:36:43.108530+0000","last_became_active":"2024-05-08T06:36:41.648406+0000","last_became_peered":"2024-05-08T06:36:41.648406+0000","last_unstale":"2024-05-08T06:36:43.108530+0000","last_undegraded":"2024-05-08T06:36:43.108530+0000","last_fullsized":"2024-05-08T06:36:43.108530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108653+0000","last_change":"2024-05-08T06:36:40.648721+0000","last_active":"2024-05-08T06:36:43.108653+0000","last_peered":"2024-05-08T06:36:43.108653+0000","last_clean":"2024-05-08T06:36:43.108653+0000","last_became_active":"2024-05-08T06:36:40.648429+0000","last_became_peered":"2024-05-08T06:36:40.648429+0000","last_unstale":"2024-05-08T06:36:43.108653+0000","last_undegraded":"2024-05-08T06:36:43.108653+0000","last_fullsized":"2024-05-08T06:36:43.108653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:43.792599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654290+0000","last_change":"2024-05-08T06:36:41.655289+0000","last_active":"2024-05-08T06:36:42.654290+0000","last_peered":"2024-05-08T06:36:42.654290+0000","last_clean":"2024-05-08T06:36:42.654290+0000","last_became_active":"2024-05-08T06:36:41.655022+0000","last_became_peered":"2024-05-08T06:36:41.655022+0000","last_unstale":"2024-05-08T06:36:42.654290+0000","last_undegraded":"2024-05-08T06:36:42.654290+0000","last_fullsized":"2024-05-08T06:36:42.654290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:57.405934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649256+0000","last_change":"2024-05-08T06:36:40.645335+0000","last_active":"2024-05-08T06:36:42.649256+0000","last_peered":"2024-05-08T06:36:42.649256+0000","last_clean":"2024-05-08T06:36:42.649256+0000","last_became_active":"2024-05-08T06:36:40.645019+0000","last_became_peered":"2024-05-08T06:36:40.645019+0000","last_unstale":"2024-05-08T06:36:42.649256+0000","last_undegraded":"2024-05-08T06:36:42.649256+0000","last_fullsized":"2024-05-08T06:36:42.649256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108851+0000","last_change":"2024-05-08T06:36:41.653910+0000","last_active":"2024-05-08T06:36:43.108851+0000","last_peered":"2024-05-08T06:36:43.108851+0000","last_clean":"2024-05-08T06:36:43.108851+0000","last_became_active":"2024-05-08T06:36:41.653567+0000","last_became_peered":"2024-05-08T06:36:41.653567+0000","last_unstale":"2024-05-08T06:36:43.108851+0000","last_undegraded":"2024-05-08T06:36:43.108851+0000","last_fullsized":"2024-05-08T06:36:43.108851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649426+0000","last_change":"2024-05-08T06:36:40.642880+0000","last_active":"2024-05-08T06:36:42.649426+0000","last_peered":"2024-05-08T06:36:42.649426+0000","last_clean":"2024-05-08T06:36:42.649426+0000","last_became_active":"2024-05-08T06:36:40.642605+0000","last_became_peered":"2024-05-08T06:36:40.642605+0000","last_unstale":"2024-05-08T06:36:42.649426+0000","last_undegraded":"2024-05-08T06:36:42.649426+0000","last_fullsized":"2024-05-08T06:36:42.649426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109118+0000","last_change":"2024-05-08T06:36:41.654124+0000","last_active":"2024-05-08T06:36:43.109118+0000","last_peered":"2024-05-08T06:36:43.109118+0000","last_clean":"2024-05-08T06:36:43.109118+0000","last_became_active":"2024-05-08T06:36:41.653837+0000","last_became_peered":"2024-05-08T06:36:41.653837+0000","last_unstale":"2024-05-08T06:36:43.109118+0000","last_undegraded":"2024-05-08T06:36:43.109118+0000","last_fullsized":"2024-05-08T06:36:43.109118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109611+0000","last_change":"2024-05-08T06:36:40.644938+0000","last_active":"2024-05-08T06:36:43.109611+0000","last_peered":"2024-05-08T06:36:43.109611+0000","last_clean":"2024-05-08T06:36:43.109611+0000","last_became_active":"2024-05-08T06:36:40.644549+0000","last_became_peered":"2024-05-08T06:36:40.644549+0000","last_unstale":"2024-05-08T06:36:43.109611+0000","last_undegraded":"2024-05-08T06:36:43.109611+0000","last_fullsized":"2024-05-08T06:36:43.109611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:50.353194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653954+0000","last_change":"2024-05-08T06:36:41.659401+0000","last_active":"2024-05-08T06:36:42.653954+0000","last_peered":"2024-05-08T06:36:42.653954+0000","last_clean":"2024-05-08T06:36:42.653954+0000","last_became_active":"2024-05-08T06:36:41.659063+0000","last_became_peered":"2024-05-08T06:36:41.659063+0000","last_unstale":"2024-05-08T06:36:42.653954+0000","last_undegraded":"2024-05-08T06:36:42.653954+0000","last_fullsized":"2024-05-08T06:36:42.653954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109449+0000","last_change":"2024-05-08T06:36:41.655858+0000","last_active":"2024-05-08T06:36:43.109449+0000","last_peered":"2024-05-08T06:36:43.109449+0000","last_clean":"2024-05-08T06:36:43.109449+0000","last_became_active":"2024-05-08T06:36:41.655569+0000","last_became_peered":"2024-05-08T06:36:41.655569+0000","last_unstale":"2024-05-08T06:36:43.109449+0000","last_undegraded":"2024-05-08T06:36:43.109449+0000","last_fullsized":"2024-05-08T06:36:43.109449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:02.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109476+0000","last_change":"2024-05-08T06:36:40.646643+0000","last_active":"2024-05-08T06:36:43.109476+0000","last_peered":"2024-05-08T06:36:43.109476+0000","last_clean":"2024-05-08T06:36:43.109476+0000","last_became_active":"2024-05-08T06:36:40.646319+0000","last_became_peered":"2024-05-08T06:36:40.646319+0000","last_unstale":"2024-05-08T06:36:43.109476+0000","last_undegraded":"2024-05-08T06:36:43.109476+0000","last_fullsized":"2024-05-08T06:36:43.109476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:03.574076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109441+0000","last_change":"2024-05-08T06:36:41.656383+0000","last_active":"2024-05-08T06:36:43.109441+0000","last_peered":"2024-05-08T06:36:43.109441+0000","last_clean":"2024-05-08T06:36:43.109441+0000","last_became_active":"2024-05-08T06:36:41.656059+0000","last_became_peered":"2024-05-08T06:36:41.656059+0000","last_unstale":"2024-05-08T06:36:43.109441+0000","last_undegraded":"2024-05-08T06:36:43.109441+0000","last_fullsized":"2024-05-08T06:36:43.109441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:49.740862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109265+0000","last_change":"2024-05-08T06:36:40.646560+0000","last_active":"2024-05-08T06:36:43.109265+0000","last_peered":"2024-05-08T06:36:43.109265+0000","last_clean":"2024-05-08T06:36:43.109265+0000","last_became_active":"2024-05-08T06:36:40.646285+0000","last_became_peered":"2024-05-08T06:36:40.646285+0000","last_unstale":"2024-05-08T06:36:43.109265+0000","last_undegraded":"2024-05-08T06:36:43.109265+0000","last_fullsized":"2024-05-08T06:36:43.109265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108439+0000","last_change":"2024-05-08T06:36:41.661975+0000","last_active":"2024-05-08T06:36:43.108439+0000","last_peered":"2024-05-08T06:36:43.108439+0000","last_clean":"2024-05-08T06:36:43.108439+0000","last_became_active":"2024-05-08T06:36:41.661759+0000","last_became_peered":"2024-05-08T06:36:41.661759+0000","last_unstale":"2024-05-08T06:36:43.108439+0000","last_undegraded":"2024-05-08T06:36:43.108439+0000","last_fullsized":"2024-05-08T06:36:43.108439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109215+0000","last_change":"2024-05-08T06:36:40.645426+0000","last_active":"2024-05-08T06:36:43.109215+0000","last_peered":"2024-05-08T06:36:43.109215+0000","last_clean":"2024-05-08T06:36:43.109215+0000","last_became_active":"2024-05-08T06:36:40.645194+0000","last_became_peered":"2024-05-08T06:36:40.645194+0000","last_unstale":"2024-05-08T06:36:43.109215+0000","last_undegraded":"2024-05-08T06:36:43.109215+0000","last_fullsized":"2024-05-08T06:36:43.109215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109153+0000","last_change":"2024-05-08T06:36:41.660148+0000","last_active":"2024-05-08T06:36:43.109153+0000","last_peered":"2024-05-08T06:36:43.109153+0000","last_clean":"2024-05-08T06:36:43.109153+0000","last_became_active":"2024-05-08T06:36:41.659907+0000","last_became_peered":"2024-05-08T06:36:41.659907+0000","last_unstale":"2024-05-08T06:36:43.109153+0000","last_undegraded":"2024-05-08T06:36:43.109153+0000","last_fullsized":"2024-05-08T06:36:43.109153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:45:07.961465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109158+0000","last_change":"2024-05-08T06:36:40.645379+0000","last_active":"2024-05-08T06:36:43.109158+0000","last_peered":"2024-05-08T06:36:43.109158+0000","last_clean":"2024-05-08T06:36:43.109158+0000","last_became_active":"2024-05-08T06:36:40.645091+0000","last_became_peered":"2024-05-08T06:36:40.645091+0000","last_unstale":"2024-05-08T06:36:43.109158+0000","last_undegraded":"2024-05-08T06:36:43.109158+0000","last_fullsized":"2024-05-08T06:36:43.109158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:36:46.336 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T06:36:46.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.472+0000 7fdcd9a72640 1 Processor -- start 2024-05-08T06:36:46.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.472+0000 7fdcd9a72640 1 -- start start 2024-05-08T06:36:46.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.473+0000 7fdcd9a72640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd406c320 0x7fdcd40b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.473+0000 7fdcd9a72640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdcd4163a60 0x7fdcd406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.473+0000 7fdcd9a72640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcd406cc40 con 0x7fdcd4160350 2024-05-08T06:36:46.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.473+0000 7fdcd9a72640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fdcd415fec0 con 0x7fdcd4163a60 2024-05-08T06:36:46.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.473+0000 7fdcd9a72640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd4163ee0 con 0x7fdcd406c320 2024-05-08T06:36:46.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.473+0000 7fdccbfff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdcd4163a60 0x7fdcd406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.473+0000 7fdccbfff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdcd4163a60 0x7fdcd406c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35556/0 (socket says 172.21.15.163:35556) 2024-05-08T06:36:46.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.473+0000 7fdccbfff640 1 -- 172.21.15.163:0/1750787869 learned_addr learned my addr 172.21.15.163:0/1750787869 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:46.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.473+0000 7fdcd3fff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fdcd4160350 0x7fdcd406f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:59890/0 (socket says 172.21.15.163:59890) 2024-05-08T06:36:46.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.473+0000 7fdcd37fe640 1 --2- 172.21.15.163:0/1750787869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd406c320 0x7fdcd40b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.473+0000 7fdccbfff640 1 -- 172.21.15.163:0/1750787869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd406c320 msgr2=0x7fdcd40b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:46.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.473+0000 7fdccbfff640 1 --2- 172.21.15.163:0/1750787869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd406c320 0x7fdcd40b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.473+0000 7fdccbfff640 1 -- 172.21.15.163:0/1750787869 >> v1:172.21.15.163:6789/0 conn(0x7fdcd4160350 legacy=0x7fdcd406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.473+0000 7fdccbfff640 1 -- 172.21.15.163:0/1750787869 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd40b5060 con 0x7fdcd4163a60 2024-05-08T06:36:46.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.474+0000 7fdccbfff640 1 --2- 172.21.15.163:0/1750787869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd4163a60 0x7fdcd406c700 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009870 tx=0x7fdcbc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91d9f9eab414c09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:46.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.474+0000 7fdcd2ffd640 1 -- 172.21.15.163:0/1750787869 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc037070 con 0x7fdcd4163a60 2024-05-08T06:36:46.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.474+0000 7fdcd2ffd640 1 -- 172.21.15.163:0/1750787869 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcbc032440 con 0x7fdcd4163a60 2024-05-08T06:36:46.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.474+0000 7fdcd2ffd640 1 -- 172.21.15.163:0/1750787869 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc032810 con 0x7fdcd4163a60 2024-05-08T06:36:46.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.474+0000 7fdcd9a72640 1 -- 172.21.15.163:0/1750787869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd4163a60 msgr2=0x7fdcd406c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.474+0000 7fdcd9a72640 1 --2- 172.21.15.163:0/1750787869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd4163a60 0x7fdcd406c700 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009870 tx=0x7fdcbc02a1c0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.475+0000 7fdcd9a72640 1 -- 172.21.15.163:0/1750787869 shutdown_connections 2024-05-08T06:36:46.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.475+0000 7fdcd9a72640 1 --2- 172.21.15.163:0/1750787869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd4163a60 0x7fdcd406c700 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.475+0000 7fdcd9a72640 1 --2- 172.21.15.163:0/1750787869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd406c320 0x7fdcd40b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.475+0000 7fdcd9a72640 1 -- 172.21.15.163:0/1750787869 >> 172.21.15.163:0/1750787869 conn(0x7fdcd40b3c30 msgr2=0x7fdcd40b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:46.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.475+0000 7fdcd9a72640 1 -- 172.21.15.163:0/1750787869 shutdown_connections 2024-05-08T06:36:46.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.475+0000 7fdcd9a72640 1 -- 172.21.15.163:0/1750787869 wait complete. 2024-05-08T06:36:46.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.476+0000 7fdcd9a72640 1 Processor -- start 2024-05-08T06:36:46.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.476+0000 7fdcd9a72640 1 -- start start 2024-05-08T06:36:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.476+0000 7fdcd9a72640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd406c320 0x7fdcd4146e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd9a72640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd4160350 0x7fdcd4148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd9a72640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd4163a60 0x7fdcd41f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd9a72640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd415fa80 con 0x7fdcd406c320 2024-05-08T06:36:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd9a72640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd40b5620 con 0x7fdcd4160350 2024-05-08T06:36:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd9a72640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd4147440 con 0x7fdcd4163a60 2024-05-08T06:36:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd37fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd406c320 0x7fdcd4146e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd37fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd406c320 0x7fdcd4146e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35562/0 (socket says 172.21.15.163:35562) 2024-05-08T06:36:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd37fe640 1 -- 172.21.15.163:0/585030642 learned_addr learned my addr 172.21.15.163:0/585030642 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd37fe640 1 -- 172.21.15.163:0/585030642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd4163a60 msgr2=0x7fdcd41f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd37fe640 1 --2- 172.21.15.163:0/585030642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd4163a60 0x7fdcd41f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdccbfff640 1 --2- 172.21.15.163:0/585030642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd4160350 0x7fdcd4148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd37fe640 1 -- 172.21.15.163:0/585030642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd4160350 msgr2=0x7fdcd4148bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd37fe640 1 --2- 172.21.15.163:0/585030642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd4160350 0x7fdcd4148bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd37fe640 1 -- 172.21.15.163:0/585030642 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd4147720 con 0x7fdcd406c320 2024-05-08T06:36:46.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdccbfff640 1 --2- 172.21.15.163:0/585030642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd4160350 0x7fdcd4148bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:46.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd37fe640 1 --2- 172.21.15.163:0/585030642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd406c320 0x7fdcd4146e10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400d5f0 tx=0x7fdcc400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:46.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd0ff9640 1 -- 172.21.15.163:0/585030642 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40125f0 con 0x7fdcd406c320 2024-05-08T06:36:46.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd0ff9640 1 -- 172.21.15.163:0/585030642 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc4019070 con 0x7fdcd406c320 2024-05-08T06:36:46.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd9a72640 1 -- 172.21.15.163:0/585030642 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd4147a00 con 0x7fdcd406c320 2024-05-08T06:36:46.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd0ff9640 1 -- 172.21.15.163:0/585030642 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40129d0 con 0x7fdcd406c320 2024-05-08T06:36:46.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.477+0000 7fdcd9a72640 1 -- 172.21.15.163:0/585030642 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd41490f0 con 0x7fdcd406c320 2024-05-08T06:36:46.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.478+0000 7fdcd9a72640 1 -- 172.21.15.163:0/585030642 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd406a4b0 con 0x7fdcd406c320 2024-05-08T06:36:46.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.479+0000 7fdcd0ff9640 1 -- 172.21.15.163:0/585030642 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdcc4012b70 con 0x7fdcd406c320 2024-05-08T06:36:46.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.479+0000 7fdcd0ff9640 1 --2- 172.21.15.163:0/585030642 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fdcb4076300 0x7fdcb40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.480+0000 7fdccbfff640 1 --2- 172.21.15.163:0/585030642 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fdcb4076300 0x7fdcb40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.480+0000 7fdcd0ff9640 1 -- 172.21.15.163:0/585030642 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdcc40979d0 con 0x7fdcd406c320 2024-05-08T06:36:46.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.480+0000 7fdccbfff640 1 --2- 172.21.15.163:0/585030642 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fdcb4076300 0x7fdcb40787c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc034000 tx=0x7fdcbc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:46.483 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.484+0000 7fdcd0ff9640 1 -- 172.21.15.163:0/585030642 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdcc40613c0 con 0x7fdcd406c320 2024-05-08T06:36:46.615 INFO:tasks.ceph.mds.d.smithi192.stderr:2024-05-08T06:36:46.616+0000 7ff9e18cc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T06:36:46.738 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.738+0000 7fdcd9a72640 1 -- 172.21.15.163:0/585030642 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fdcd406ce50 con 0x7fdcd406c320 2024-05-08T06:36:46.738 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.739+0000 7fdcd0ff9640 1 -- 172.21.15.163:0/585030642 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1011 (secure 0 0 0) 0x7fdcc4060cc0 con 0x7fdcd406c320 2024-05-08T06:36:46.738 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:46.738 INFO:teuthology.orchestra.run.smithi163.stdout:{"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":28,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1715150189,"num_in_osds":8,"osd_in_since":1715150180,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":224239616,"bytes_avail":772869873664,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":7,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4300}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-05-08T06:36:17.751637+0000","services":{}},"progress_events":{}} 2024-05-08T06:36:46.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.741+0000 7fdcd9a72640 1 -- 172.21.15.163:0/585030642 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fdcb4076300 msgr2=0x7fdcb40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.741+0000 7fdcd9a72640 1 --2- 172.21.15.163:0/585030642 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fdcb4076300 0x7fdcb40787c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc034000 tx=0x7fdcbc02e000 comp rx=0 tx=0).stop 2024-05-08T06:36:46.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.741+0000 7fdcd9a72640 1 -- 172.21.15.163:0/585030642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd406c320 msgr2=0x7fdcd4146e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.741 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.741+0000 7fdcd9a72640 1 --2- 172.21.15.163:0/585030642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd406c320 0x7fdcd4146e10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400d5f0 tx=0x7fdcc400dab0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.741 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.741+0000 7fdcd9a72640 1 -- 172.21.15.163:0/585030642 shutdown_connections 2024-05-08T06:36:46.741 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.741+0000 7fdcd9a72640 1 --2- 172.21.15.163:0/585030642 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fdcb4076300 0x7fdcb40787c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.741 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.741+0000 7fdcd9a72640 1 --2- 172.21.15.163:0/585030642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd4163a60 0x7fdcd41f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.741 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.741+0000 7fdcd9a72640 1 --2- 172.21.15.163:0/585030642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd4160350 0x7fdcd4148bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.741 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.741+0000 7fdcd9a72640 1 --2- 172.21.15.163:0/585030642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd406c320 0x7fdcd4146e10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.741 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.741+0000 7fdcd9a72640 1 -- 172.21.15.163:0/585030642 >> 172.21.15.163:0/585030642 conn(0x7fdcd40b3c30 msgr2=0x7fdcd40b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:46.741 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.741+0000 7fdcd9a72640 1 -- 172.21.15.163:0/585030642 shutdown_connections 2024-05-08T06:36:46.741 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.741+0000 7fdcd9a72640 1 -- 172.21.15.163:0/585030642 wait complete. 2024-05-08T06:36:46.751 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-08T06:36:46.751 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:36:46.851 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.852+0000 7f9347d5f640 1 Processor -- start 2024-05-08T06:36:46.851 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.852+0000 7f9347d5f640 1 -- start start 2024-05-08T06:36:46.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9347d5f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f93400aef80 0x7f9340073f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9347d5f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93400d2170 0x7f93400d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9347d5f640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93400b74f0 con 0x7f9340162900 2024-05-08T06:36:46.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9347d5f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f93400d2b00 con 0x7f93400aef80 2024-05-08T06:36:46.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9347d5f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9340162ce0 con 0x7f93400d2170 2024-05-08T06:36:46.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9345afb640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f9340162900 0x7f93400b6ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:59892/0 (socket says 172.21.15.163:59892) 2024-05-08T06:36:46.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9345afb640 1 -- 172.21.15.163:0/1158969762 learned_addr learned my addr 172.21.15.163:0/1158969762 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:46.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f93452fa640 1 --2- 172.21.15.163:0/1158969762 >> v2:172.21.15.192:3300/0 conn(0x7f93400aef80 0x7f9340073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f93462fc640 1 --2- 172.21.15.163:0/1158969762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93400d2170 0x7f93400d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9344af9640 1 -- 172.21.15.163:0/1158969762 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1779463797 0 0) 0x7f93400b74f0 con 0x7f9340162900 2024-05-08T06:36:46.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9344af9640 1 -- 172.21.15.163:0/1158969762 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f932c003520 con 0x7f9340162900 2024-05-08T06:36:46.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9344af9640 1 -- 172.21.15.163:0/1158969762 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2060017355 0 0) 0x7f932c003520 con 0x7f9340162900 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9344af9640 1 -- 172.21.15.163:0/1158969762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93400d2170 msgr2=0x7f93400d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9344af9640 1 --2- 172.21.15.163:0/1158969762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93400d2170 0x7f93400d25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9344af9640 1 -- 172.21.15.163:0/1158969762 >> v2:172.21.15.192:3300/0 conn(0x7f93400aef80 msgr2=0x7f9340073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9344af9640 1 --2- 172.21.15.163:0/1158969762 >> v2:172.21.15.192:3300/0 conn(0x7f93400aef80 0x7f9340073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9344af9640 1 -- 172.21.15.163:0/1158969762 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93400b50b0 con 0x7f9340162900 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9344af9640 1 -- 172.21.15.163:0/1158969762 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f933c0016f0 con 0x7f9340162900 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f93452fa640 1 --2- 172.21.15.163:0/1158969762 >> v2:172.21.15.192:3300/0 conn(0x7f93400aef80 0x7f9340073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9344af9640 1 -- 172.21.15.163:0/1158969762 >> v1:172.21.15.163:6789/0 conn(0x7f9340162900 legacy=0x7f93400b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.853+0000 7f9344af9640 1 --2- 172.21.15.163:0/1158969762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f932c003a50 0x7f932c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.854+0000 7f9344af9640 1 --2- 172.21.15.163:0/1158969762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93400d2170 0x7f932c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.854+0000 7f9344af9640 1 --2- 172.21.15.163:0/1158969762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f932c004d40 0x7f932c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.854+0000 7f9344af9640 1 -- 172.21.15.163:0/1158969762 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f93400d2b00 con 0x7f932c003a50 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.854+0000 7f9344af9640 1 -- 172.21.15.163:0/1158969762 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9340162ce0 con 0x7f932c004d40 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.854+0000 7f9344af9640 1 -- 172.21.15.163:0/1158969762 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f932c00ba90 con 0x7f93400d2170 2024-05-08T06:36:46.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.854+0000 7f9345afb640 1 --2- 172.21.15.163:0/1158969762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f932c003a50 0x7f932c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.854+0000 7f93452fa640 1 --2- 172.21.15.163:0/1158969762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93400d2170 0x7f932c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.854+0000 7f93462fc640 1 --2- 172.21.15.163:0/1158969762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f932c004d40 0x7f932c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.854+0000 7f9345afb640 1 -- 172.21.15.163:0/1158969762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93400d2170 msgr2=0x7f932c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.854+0000 7f9345afb640 1 --2- 172.21.15.163:0/1158969762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93400d2170 0x7f932c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.854+0000 7f9345afb640 1 -- 172.21.15.163:0/1158969762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f932c004d40 msgr2=0x7f932c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.854+0000 7f9345afb640 1 --2- 172.21.15.163:0/1158969762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f932c004d40 0x7f932c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.854+0000 7f9345afb640 1 -- 172.21.15.163:0/1158969762 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c003520 con 0x7f932c003a50 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.854+0000 7f93462fc640 1 --2- 172.21.15.163:0/1158969762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f932c004d40 0x7f932c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.855+0000 7f9345afb640 1 --2- 172.21.15.163:0/1158969762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f932c003a50 0x7f932c003e70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f933c00c210 tx=0x7f932c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36455863ac4773e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.855+0000 7f9344af9640 1 -- 172.21.15.163:0/1158969762 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c007d30 con 0x7f932c003a50 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.855+0000 7f9344af9640 1 -- 172.21.15.163:0/1158969762 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933c02e070 con 0x7f932c003a50 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.855+0000 7f9344af9640 1 -- 172.21.15.163:0/1158969762 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c005d30 con 0x7f932c003a50 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.855+0000 7f9347d5f640 1 -- 172.21.15.163:0/1158969762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f932c003a50 msgr2=0x7f932c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.855+0000 7f9347d5f640 1 --2- 172.21.15.163:0/1158969762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f932c003a50 0x7f932c003e70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f933c00c210 tx=0x7f932c005830 comp rx=0 tx=0).stop 2024-05-08T06:36:46.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.855+0000 7f93462fc640 1 -- 172.21.15.163:0/1158969762 reap_dead start 2024-05-08T06:36:46.858 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.855+0000 7f9347d5f640 1 -- 172.21.15.163:0/1158969762 shutdown_connections 2024-05-08T06:36:46.858 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.855+0000 7f9347d5f640 1 -- 172.21.15.163:0/1158969762 >> 172.21.15.163:0/1158969762 conn(0x7f934006d610 msgr2=0x7f934006d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:46.858 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.855+0000 7f9347d5f640 1 -- 172.21.15.163:0/1158969762 shutdown_connections 2024-05-08T06:36:46.858 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.855+0000 7f9347d5f640 1 -- 172.21.15.163:0/1158969762 wait complete. 2024-05-08T06:36:46.860 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.856+0000 7f9347d5f640 1 Processor -- start 2024-05-08T06:36:46.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.857+0000 7f9347d5f640 1 -- start start 2024-05-08T06:36:46.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.857+0000 7f9347d5f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f932c003a50 0x7f93400c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.857+0000 7f9347d5f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93400d0be0 0x7f93400af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.857+0000 7f9347d5f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f934015b870 0x7f934015bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.857+0000 7f9347d5f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f934006f4d0 con 0x7f932c003a50 2024-05-08T06:36:46.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.857+0000 7f9347d5f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f93400b5670 con 0x7f934015b870 2024-05-08T06:36:46.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.857+0000 7f9347d5f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f93400d25c0 con 0x7f93400d0be0 2024-05-08T06:36:46.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.857+0000 7f9345afb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f932c003a50 0x7f93400c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.857+0000 7f9345afb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f932c003a50 0x7f93400c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35570/0 (socket says 172.21.15.163:35570) 2024-05-08T06:36:46.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.857+0000 7f9345afb640 1 -- 172.21.15.163:0/1903181473 learned_addr learned my addr 172.21.15.163:0/1903181473 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:46.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.857+0000 7f93462fc640 1 --2- 172.21.15.163:0/1903181473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f934015b870 0x7f934015bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.857+0000 7f93452fa640 1 --2- 172.21.15.163:0/1903181473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93400d0be0 0x7f93400af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.858+0000 7f9345afb640 1 -- 172.21.15.163:0/1903181473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93400d0be0 msgr2=0x7f93400af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.858+0000 7f9345afb640 1 --2- 172.21.15.163:0/1903181473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93400d0be0 0x7f93400af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.858+0000 7f9345afb640 1 -- 172.21.15.163:0/1903181473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f934015b870 msgr2=0x7f934015bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:46.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.858+0000 7f9345afb640 1 --2- 172.21.15.163:0/1903181473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f934015b870 0x7f934015bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:46.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.858+0000 7f9345afb640 1 -- 172.21.15.163:0/1903181473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93400c74f0 con 0x7f932c003a50 2024-05-08T06:36:46.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.858+0000 7f9345afb640 1 --2- 172.21.15.163:0/1903181473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f932c003a50 0x7f93400c6f10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f933c00c330 tx=0x7f93400b4fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:46.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.858+0000 7f93367fc640 1 -- 172.21.15.163:0/1903181473 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c007480 con 0x7f932c003a50 2024-05-08T06:36:46.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.858+0000 7f93367fc640 1 -- 172.21.15.163:0/1903181473 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933c007620 con 0x7f932c003a50 2024-05-08T06:36:46.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.858+0000 7f9347d5f640 1 -- 172.21.15.163:0/1903181473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93400c77d0 con 0x7f932c003a50 2024-05-08T06:36:46.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.858+0000 7f93367fc640 1 -- 172.21.15.163:0/1903181473 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c033670 con 0x7f932c003a50 2024-05-08T06:36:46.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.858+0000 7f9347d5f640 1 -- 172.21.15.163:0/1903181473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93400c7d90 con 0x7f932c003a50 2024-05-08T06:36:46.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.859+0000 7f9347d5f640 1 -- 172.21.15.163:0/1903181473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f934006df20 con 0x7f932c003a50 2024-05-08T06:36:46.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.860+0000 7f93367fc640 1 -- 172.21.15.163:0/1903181473 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f933c03e030 con 0x7f932c003a50 2024-05-08T06:36:46.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.860+0000 7f93367fc640 1 --2- 172.21.15.163:0/1903181473 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f93240762d0 0x7f9324078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:46.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.860+0000 7f93452fa640 1 --2- 172.21.15.163:0/1903181473 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f93240762d0 0x7f9324078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:46.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.860+0000 7f93367fc640 1 -- 172.21.15.163:0/1903181473 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f933c0b7160 con 0x7f932c003a50 2024-05-08T06:36:46.863 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.861+0000 7f93452fa640 1 --2- 172.21.15.163:0/1903181473 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f93240762d0 0x7f9324078790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f934015b5f0 tx=0x7f9330009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:46.863 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:46.864+0000 7f93367fc640 1 -- 172.21.15.163:0/1903181473 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f934006df20 con 0x7f932c003a50 2024-05-08T06:36:47.042 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:47.043+0000 7f9347d5f640 1 -- 172.21.15.163:0/1903181473 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f934006a550 con 0x7f93240762d0 2024-05-08T06:36:47.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:47.049+0000 7f93367fc640 1 -- 172.21.15.163:0/1903181473 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7f934006a550 con 0x7f93240762d0 2024-05-08T06:36:47.051 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:47.052 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2024-05-08T06:36:47.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:47.052+0000 7f9347d5f640 1 -- 172.21.15.163:0/1903181473 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f93240762d0 msgr2=0x7f9324078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:47.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:47.052+0000 7f9347d5f640 1 --2- 172.21.15.163:0/1903181473 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f93240762d0 0x7f9324078790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f934015b5f0 tx=0x7f9330009190 comp rx=0 tx=0).stop 2024-05-08T06:36:47.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:47.052+0000 7f9347d5f640 1 -- 172.21.15.163:0/1903181473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f932c003a50 msgr2=0x7f93400c6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:47.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:47.052+0000 7f9347d5f640 1 --2- 172.21.15.163:0/1903181473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f932c003a50 0x7f93400c6f10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f933c00c330 tx=0x7f93400b4fa0 comp rx=0 tx=0).stop 2024-05-08T06:36:47.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:47.052+0000 7f9347d5f640 1 -- 172.21.15.163:0/1903181473 shutdown_connections 2024-05-08T06:36:47.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:47.052+0000 7f9347d5f640 1 --2- 172.21.15.163:0/1903181473 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f93240762d0 0x7f9324078790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:47.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:47.052+0000 7f9347d5f640 1 --2- 172.21.15.163:0/1903181473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f934015b870 0x7f934015bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:47.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:47.052+0000 7f9347d5f640 1 --2- 172.21.15.163:0/1903181473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93400d0be0 0x7f93400af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:47.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:47.052+0000 7f9347d5f640 1 --2- 172.21.15.163:0/1903181473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f932c003a50 0x7f93400c6f10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:47.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:47.052+0000 7f9347d5f640 1 -- 172.21.15.163:0/1903181473 >> 172.21.15.163:0/1903181473 conn(0x7f934006d610 msgr2=0x7f93400b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:47.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:47.052+0000 7f9347d5f640 1 -- 172.21.15.163:0/1903181473 shutdown_connections 2024-05-08T06:36:47.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:47.052+0000 7f9347d5f640 1 -- 172.21.15.163:0/1903181473 wait complete. 2024-05-08T06:36:47.062 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-05-08T06:36:46.098319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001183"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109935+0000","last_change":"2024-05-08T06:36:41.664385+0000","last_active":"2024-05-08T06:36:43.109935+0000","last_peered":"2024-05-08T06:36:43.109935+0000","last_clean":"2024-05-08T06:36:43.109935+0000","last_became_active":"2024-05-08T06:36:41.664106+0000","last_became_peered":"2024-05-08T06:36:41.664106+0000","last_unstale":"2024-05-08T06:36:43.109935+0000","last_undegraded":"2024-05-08T06:36:43.109935+0000","last_fullsized":"2024-05-08T06:36:43.109935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:08.774451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649403+0000","last_change":"2024-05-08T06:36:41.647763+0000","last_active":"2024-05-08T06:36:42.649403+0000","last_peered":"2024-05-08T06:36:42.649403+0000","last_clean":"2024-05-08T06:36:42.649403+0000","last_became_active":"2024-05-08T06:36:41.647487+0000","last_became_peered":"2024-05-08T06:36:41.647487+0000","last_unstale":"2024-05-08T06:36:42.649403+0000","last_undegraded":"2024-05-08T06:36:42.649403+0000","last_fullsized":"2024-05-08T06:36:42.649403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108764+0000","last_change":"2024-05-08T06:36:41.649945+0000","last_active":"2024-05-08T06:36:43.108764+0000","last_peered":"2024-05-08T06:36:43.108764+0000","last_clean":"2024-05-08T06:36:43.108764+0000","last_became_active":"2024-05-08T06:36:41.649665+0000","last_became_peered":"2024-05-08T06:36:41.649665+0000","last_unstale":"2024-05-08T06:36:43.108764+0000","last_undegraded":"2024-05-08T06:36:43.108764+0000","last_fullsized":"2024-05-08T06:36:43.108764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109325+0000","last_change":"2024-05-08T06:36:41.656179+0000","last_active":"2024-05-08T06:36:43.109325+0000","last_peered":"2024-05-08T06:36:43.109325+0000","last_clean":"2024-05-08T06:36:43.109325+0000","last_became_active":"2024-05-08T06:36:41.655649+0000","last_became_peered":"2024-05-08T06:36:41.655649+0000","last_unstale":"2024-05-08T06:36:43.109325+0000","last_undegraded":"2024-05-08T06:36:43.109325+0000","last_fullsized":"2024-05-08T06:36:43.109325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:45:07.961465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109228+0000","last_change":"2024-05-08T06:36:41.659928+0000","last_active":"2024-05-08T06:36:43.109228+0000","last_peered":"2024-05-08T06:36:43.109228+0000","last_clean":"2024-05-08T06:36:43.109228+0000","last_became_active":"2024-05-08T06:36:41.659408+0000","last_became_peered":"2024-05-08T06:36:41.659408+0000","last_unstale":"2024-05-08T06:36:43.109228+0000","last_undegraded":"2024-05-08T06:36:43.109228+0000","last_fullsized":"2024-05-08T06:36:43.109228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:14.886723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109064+0000","last_change":"2024-05-08T06:36:41.653658+0000","last_active":"2024-05-08T06:36:43.109064+0000","last_peered":"2024-05-08T06:36:43.109064+0000","last_clean":"2024-05-08T06:36:43.109064+0000","last_became_active":"2024-05-08T06:36:41.653285+0000","last_became_peered":"2024-05-08T06:36:41.653285+0000","last_unstale":"2024-05-08T06:36:43.109064+0000","last_undegraded":"2024-05-08T06:36:43.109064+0000","last_fullsized":"2024-05-08T06:36:43.109064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108884+0000","last_change":"2024-05-08T06:36:41.661193+0000","last_active":"2024-05-08T06:36:43.108884+0000","last_peered":"2024-05-08T06:36:43.108884+0000","last_clean":"2024-05-08T06:36:43.108884+0000","last_became_active":"2024-05-08T06:36:41.660907+0000","last_became_peered":"2024-05-08T06:36:41.660907+0000","last_unstale":"2024-05-08T06:36:43.108884+0000","last_undegraded":"2024-05-08T06:36:43.108884+0000","last_fullsized":"2024-05-08T06:36:43.108884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108493+0000","last_change":"2024-05-08T06:36:41.655350+0000","last_active":"2024-05-08T06:36:43.108493+0000","last_peered":"2024-05-08T06:36:43.108493+0000","last_clean":"2024-05-08T06:36:43.108493+0000","last_became_active":"2024-05-08T06:36:41.655064+0000","last_became_peered":"2024-05-08T06:36:41.655064+0000","last_unstale":"2024-05-08T06:36:43.108493+0000","last_undegraded":"2024-05-08T06:36:43.108493+0000","last_fullsized":"2024-05-08T06:36:43.108493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:11:54.263628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109972+0000","last_change":"2024-05-08T06:36:41.663210+0000","last_active":"2024-05-08T06:36:43.109972+0000","last_peered":"2024-05-08T06:36:43.109972+0000","last_clean":"2024-05-08T06:36:43.109972+0000","last_became_active":"2024-05-08T06:36:41.662954+0000","last_became_peered":"2024-05-08T06:36:41.662954+0000","last_unstale":"2024-05-08T06:36:43.109972+0000","last_undegraded":"2024-05-08T06:36:43.109972+0000","last_fullsized":"2024-05-08T06:36:43.109972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:36:32.652292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654330+0000","last_change":"2024-05-08T06:36:41.662910+0000","last_active":"2024-05-08T06:36:42.654330+0000","last_peered":"2024-05-08T06:36:42.654330+0000","last_clean":"2024-05-08T06:36:42.654330+0000","last_became_active":"2024-05-08T06:36:41.662585+0000","last_became_peered":"2024-05-08T06:36:41.662585+0000","last_unstale":"2024-05-08T06:36:42.654330+0000","last_undegraded":"2024-05-08T06:36:42.654330+0000","last_fullsized":"2024-05-08T06:36:42.654330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653737+0000","last_change":"2024-05-08T06:36:41.663863+0000","last_active":"2024-05-08T06:36:42.653737+0000","last_peered":"2024-05-08T06:36:42.653737+0000","last_clean":"2024-05-08T06:36:42.653737+0000","last_became_active":"2024-05-08T06:36:41.663584+0000","last_became_peered":"2024-05-08T06:36:41.663584+0000","last_unstale":"2024-05-08T06:36:42.653737+0000","last_undegraded":"2024-05-08T06:36:42.653737+0000","last_fullsized":"2024-05-08T06:36:42.653737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654617+0000","last_change":"2024-05-08T06:36:41.661979+0000","last_active":"2024-05-08T06:36:42.654617+0000","last_peered":"2024-05-08T06:36:42.654617+0000","last_clean":"2024-05-08T06:36:42.654617+0000","last_became_active":"2024-05-08T06:36:41.660222+0000","last_became_peered":"2024-05-08T06:36:41.660222+0000","last_unstale":"2024-05-08T06:36:42.654617+0000","last_undegraded":"2024-05-08T06:36:42.654617+0000","last_fullsized":"2024-05-08T06:36:42.654617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:26.842934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108871+0000","last_change":"2024-05-08T06:36:41.654832+0000","last_active":"2024-05-08T06:36:43.108871+0000","last_peered":"2024-05-08T06:36:43.108871+0000","last_clean":"2024-05-08T06:36:43.108871+0000","last_became_active":"2024-05-08T06:36:41.654550+0000","last_became_peered":"2024-05-08T06:36:41.654550+0000","last_unstale":"2024-05-08T06:36:43.108871+0000","last_undegraded":"2024-05-08T06:36:43.108871+0000","last_fullsized":"2024-05-08T06:36:43.108871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108766+0000","last_change":"2024-05-08T06:36:41.642429+0000","last_active":"2024-05-08T06:36:43.108766+0000","last_peered":"2024-05-08T06:36:43.108766+0000","last_clean":"2024-05-08T06:36:43.108766+0000","last_became_active":"2024-05-08T06:36:41.641956+0000","last_became_peered":"2024-05-08T06:36:41.641956+0000","last_unstale":"2024-05-08T06:36:43.108766+0000","last_undegraded":"2024-05-08T06:36:43.108766+0000","last_fullsized":"2024-05-08T06:36:43.108766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649200+0000","last_change":"2024-05-08T06:36:41.656423+0000","last_active":"2024-05-08T06:36:42.649200+0000","last_peered":"2024-05-08T06:36:42.649200+0000","last_clean":"2024-05-08T06:36:42.649200+0000","last_became_active":"2024-05-08T06:36:41.656154+0000","last_became_peered":"2024-05-08T06:36:41.656154+0000","last_unstale":"2024-05-08T06:36:42.649200+0000","last_undegraded":"2024-05-08T06:36:42.649200+0000","last_fullsized":"2024-05-08T06:36:42.649200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109681+0000","last_change":"2024-05-08T06:36:41.653378+0000","last_active":"2024-05-08T06:36:43.109681+0000","last_peered":"2024-05-08T06:36:43.109681+0000","last_clean":"2024-05-08T06:36:43.109681+0000","last_became_active":"2024-05-08T06:36:41.653027+0000","last_became_peered":"2024-05-08T06:36:41.653027+0000","last_unstale":"2024-05-08T06:36:43.109681+0000","last_undegraded":"2024-05-08T06:36:43.109681+0000","last_fullsized":"2024-05-08T06:36:43.109681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:02.654155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109919+0000","last_change":"2024-05-08T06:36:41.659055+0000","last_active":"2024-05-08T06:36:43.109919+0000","last_peered":"2024-05-08T06:36:43.109919+0000","last_clean":"2024-05-08T06:36:43.109919+0000","last_became_active":"2024-05-08T06:36:41.658774+0000","last_became_peered":"2024-05-08T06:36:41.658774+0000","last_unstale":"2024-05-08T06:36:43.109919+0000","last_undegraded":"2024-05-08T06:36:43.109919+0000","last_fullsized":"2024-05-08T06:36:43.109919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:38:12.052561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110931+0000","last_change":"2024-05-08T06:36:41.649669+0000","last_active":"2024-05-08T06:36:43.110931+0000","last_peered":"2024-05-08T06:36:43.110931+0000","last_clean":"2024-05-08T06:36:43.110931+0000","last_became_active":"2024-05-08T06:36:41.649391+0000","last_became_peered":"2024-05-08T06:36:41.649391+0000","last_unstale":"2024-05-08T06:36:43.110931+0000","last_undegraded":"2024-05-08T06:36:43.110931+0000","last_fullsized":"2024-05-08T06:36:43.110931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:04.579668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109443+0000","last_change":"2024-05-08T06:36:41.660852+0000","last_active":"2024-05-08T06:36:43.109443+0000","last_peered":"2024-05-08T06:36:43.109443+0000","last_clean":"2024-05-08T06:36:43.109443+0000","last_became_active":"2024-05-08T06:36:41.660573+0000","last_became_peered":"2024-05-08T06:36:41.660573+0000","last_unstale":"2024-05-08T06:36:43.109443+0000","last_undegraded":"2024-05-08T06:36:43.109443+0000","last_fullsized":"2024-05-08T06:36:43.109443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:02.654155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653881+0000","last_change":"2024-05-08T06:36:41.663211+0000","last_active":"2024-05-08T06:36:42.653881+0000","last_peered":"2024-05-08T06:36:42.653881+0000","last_clean":"2024-05-08T06:36:42.653881+0000","last_became_active":"2024-05-08T06:36:41.662936+0000","last_became_peered":"2024-05-08T06:36:41.662936+0000","last_unstale":"2024-05-08T06:36:42.653881+0000","last_undegraded":"2024-05-08T06:36:42.653881+0000","last_fullsized":"2024-05-08T06:36:42.653881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108430+0000","last_change":"2024-05-08T06:36:41.661453+0000","last_active":"2024-05-08T06:36:43.108430+0000","last_peered":"2024-05-08T06:36:43.108430+0000","last_clean":"2024-05-08T06:36:43.108430+0000","last_became_active":"2024-05-08T06:36:41.661146+0000","last_became_peered":"2024-05-08T06:36:41.661146+0000","last_unstale":"2024-05-08T06:36:43.108430+0000","last_undegraded":"2024-05-08T06:36:43.108430+0000","last_fullsized":"2024-05-08T06:36:43.108430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:11:54.263628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109377+0000","last_change":"2024-05-08T06:36:41.658664+0000","last_active":"2024-05-08T06:36:43.109377+0000","last_peered":"2024-05-08T06:36:43.109377+0000","last_clean":"2024-05-08T06:36:43.109377+0000","last_became_active":"2024-05-08T06:36:41.658352+0000","last_became_peered":"2024-05-08T06:36:41.658352+0000","last_unstale":"2024-05-08T06:36:43.109377+0000","last_undegraded":"2024-05-08T06:36:43.109377+0000","last_fullsized":"2024-05-08T06:36:43.109377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108694+0000","last_change":"2024-05-08T06:36:41.658967+0000","last_active":"2024-05-08T06:36:43.108694+0000","last_peered":"2024-05-08T06:36:43.108694+0000","last_clean":"2024-05-08T06:36:43.108694+0000","last_became_active":"2024-05-08T06:36:41.658387+0000","last_became_peered":"2024-05-08T06:36:41.658387+0000","last_unstale":"2024-05-08T06:36:43.108694+0000","last_undegraded":"2024-05-08T06:36:43.108694+0000","last_fullsized":"2024-05-08T06:36:43.108694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108892+0000","last_change":"2024-05-08T06:36:41.657102+0000","last_active":"2024-05-08T06:36:43.108892+0000","last_peered":"2024-05-08T06:36:43.108892+0000","last_clean":"2024-05-08T06:36:43.108892+0000","last_became_active":"2024-05-08T06:36:41.656844+0000","last_became_peered":"2024-05-08T06:36:41.656844+0000","last_unstale":"2024-05-08T06:36:43.108892+0000","last_undegraded":"2024-05-08T06:36:43.108892+0000","last_fullsized":"2024-05-08T06:36:43.108892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109150+0000","last_change":"2024-05-08T06:36:41.659592+0000","last_active":"2024-05-08T06:36:43.109150+0000","last_peered":"2024-05-08T06:36:43.109150+0000","last_clean":"2024-05-08T06:36:43.109150+0000","last_became_active":"2024-05-08T06:36:41.659295+0000","last_became_peered":"2024-05-08T06:36:41.659295+0000","last_unstale":"2024-05-08T06:36:43.109150+0000","last_undegraded":"2024-05-08T06:36:43.109150+0000","last_fullsized":"2024-05-08T06:36:43.109150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654479+0000","last_change":"2024-05-08T06:36:41.649572+0000","last_active":"2024-05-08T06:36:42.654479+0000","last_peered":"2024-05-08T06:36:42.654479+0000","last_clean":"2024-05-08T06:36:42.654479+0000","last_became_active":"2024-05-08T06:36:41.649302+0000","last_became_peered":"2024-05-08T06:36:41.649302+0000","last_unstale":"2024-05-08T06:36:42.654479+0000","last_undegraded":"2024-05-08T06:36:42.654479+0000","last_fullsized":"2024-05-08T06:36:42.654479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649400+0000","last_change":"2024-05-08T06:36:41.657182+0000","last_active":"2024-05-08T06:36:42.649400+0000","last_peered":"2024-05-08T06:36:42.649400+0000","last_clean":"2024-05-08T06:36:42.649400+0000","last_became_active":"2024-05-08T06:36:41.656919+0000","last_became_peered":"2024-05-08T06:36:41.656919+0000","last_unstale":"2024-05-08T06:36:42.649400+0000","last_undegraded":"2024-05-08T06:36:42.649400+0000","last_fullsized":"2024-05-08T06:36:42.649400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109121+0000","last_change":"2024-05-08T06:36:41.659147+0000","last_active":"2024-05-08T06:36:43.109121+0000","last_peered":"2024-05-08T06:36:43.109121+0000","last_clean":"2024-05-08T06:36:43.109121+0000","last_became_active":"2024-05-08T06:36:41.658880+0000","last_became_peered":"2024-05-08T06:36:41.658880+0000","last_unstale":"2024-05-08T06:36:43.109121+0000","last_undegraded":"2024-05-08T06:36:43.109121+0000","last_fullsized":"2024-05-08T06:36:43.109121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108909+0000","last_change":"2024-05-08T06:36:41.658925+0000","last_active":"2024-05-08T06:36:43.108909+0000","last_peered":"2024-05-08T06:36:43.108909+0000","last_clean":"2024-05-08T06:36:43.108909+0000","last_became_active":"2024-05-08T06:36:41.656879+0000","last_became_peered":"2024-05-08T06:36:41.656879+0000","last_unstale":"2024-05-08T06:36:43.108909+0000","last_undegraded":"2024-05-08T06:36:43.108909+0000","last_fullsized":"2024-05-08T06:36:43.108909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109551+0000","last_change":"2024-05-08T06:36:41.655125+0000","last_active":"2024-05-08T06:36:43.109551+0000","last_peered":"2024-05-08T06:36:43.109551+0000","last_clean":"2024-05-08T06:36:43.109551+0000","last_became_active":"2024-05-08T06:36:41.654817+0000","last_became_peered":"2024-05-08T06:36:41.654817+0000","last_unstale":"2024-05-08T06:36:43.109551+0000","last_undegraded":"2024-05-08T06:36:43.109551+0000","last_fullsized":"2024-05-08T06:36:43.109551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:42.444444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110786+0000","last_change":"2024-05-08T06:36:41.655361+0000","last_active":"2024-05-08T06:36:43.110786+0000","last_peered":"2024-05-08T06:36:43.110786+0000","last_clean":"2024-05-08T06:36:43.110786+0000","last_became_active":"2024-05-08T06:36:41.655085+0000","last_became_peered":"2024-05-08T06:36:41.655085+0000","last_unstale":"2024-05-08T06:36:43.110786+0000","last_undegraded":"2024-05-08T06:36:43.110786+0000","last_fullsized":"2024-05-08T06:36:43.110786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:26.842934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653569+0000","last_change":"2024-05-08T06:36:41.643791+0000","last_active":"2024-05-08T06:36:42.653569+0000","last_peered":"2024-05-08T06:36:42.653569+0000","last_clean":"2024-05-08T06:36:42.653569+0000","last_became_active":"2024-05-08T06:36:41.643506+0000","last_became_peered":"2024-05-08T06:36:41.643506+0000","last_unstale":"2024-05-08T06:36:42.653569+0000","last_undegraded":"2024-05-08T06:36:42.653569+0000","last_fullsized":"2024-05-08T06:36:42.653569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654375+0000","last_change":"2024-05-08T06:36:41.654106+0000","last_active":"2024-05-08T06:36:42.654375+0000","last_peered":"2024-05-08T06:36:42.654375+0000","last_clean":"2024-05-08T06:36:42.654375+0000","last_became_active":"2024-05-08T06:36:41.653822+0000","last_became_peered":"2024-05-08T06:36:41.653822+0000","last_unstale":"2024-05-08T06:36:42.654375+0000","last_undegraded":"2024-05-08T06:36:42.654375+0000","last_fullsized":"2024-05-08T06:36:42.654375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:51.358786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109209+0000","last_change":"2024-05-08T06:36:40.641349+0000","last_active":"2024-05-08T06:36:43.109209+0000","last_peered":"2024-05-08T06:36:43.109209+0000","last_clean":"2024-05-08T06:36:43.109209+0000","last_became_active":"2024-05-08T06:36:40.641086+0000","last_became_peered":"2024-05-08T06:36:40.641086+0000","last_unstale":"2024-05-08T06:36:43.109209+0000","last_undegraded":"2024-05-08T06:36:43.109209+0000","last_fullsized":"2024-05-08T06:36:43.109209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:39:09.616149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108579+0000","last_change":"2024-05-08T06:36:41.656120+0000","last_active":"2024-05-08T06:36:43.108579+0000","last_peered":"2024-05-08T06:36:43.108579+0000","last_clean":"2024-05-08T06:36:43.108579+0000","last_became_active":"2024-05-08T06:36:41.655843+0000","last_became_peered":"2024-05-08T06:36:41.655843+0000","last_unstale":"2024-05-08T06:36:43.108579+0000","last_undegraded":"2024-05-08T06:36:43.108579+0000","last_fullsized":"2024-05-08T06:36:43.108579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654066+0000","last_change":"2024-05-08T06:36:40.639740+0000","last_active":"2024-05-08T06:36:42.654066+0000","last_peered":"2024-05-08T06:36:42.654066+0000","last_clean":"2024-05-08T06:36:42.654066+0000","last_became_active":"2024-05-08T06:36:40.639370+0000","last_became_peered":"2024-05-08T06:36:40.639370+0000","last_unstale":"2024-05-08T06:36:42.654066+0000","last_undegraded":"2024-05-08T06:36:42.654066+0000","last_fullsized":"2024-05-08T06:36:42.654066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108684+0000","last_change":"2024-05-08T06:36:40.641983+0000","last_active":"2024-05-08T06:36:43.108684+0000","last_peered":"2024-05-08T06:36:43.108684+0000","last_clean":"2024-05-08T06:36:43.108684+0000","last_became_active":"2024-05-08T06:36:40.641666+0000","last_became_peered":"2024-05-08T06:36:40.641666+0000","last_unstale":"2024-05-08T06:36:43.108684+0000","last_undegraded":"2024-05-08T06:36:43.108684+0000","last_fullsized":"2024-05-08T06:36:43.108684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:47.997246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108516+0000","last_change":"2024-05-08T06:36:41.653257+0000","last_active":"2024-05-08T06:36:43.108516+0000","last_peered":"2024-05-08T06:36:43.108516+0000","last_clean":"2024-05-08T06:36:43.108516+0000","last_became_active":"2024-05-08T06:36:41.652900+0000","last_became_peered":"2024-05-08T06:36:41.652900+0000","last_unstale":"2024-05-08T06:36:43.108516+0000","last_undegraded":"2024-05-08T06:36:43.108516+0000","last_fullsized":"2024-05-08T06:36:43.108516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109272+0000","last_change":"2024-05-08T06:36:40.646856+0000","last_active":"2024-05-08T06:36:43.109272+0000","last_peered":"2024-05-08T06:36:43.109272+0000","last_clean":"2024-05-08T06:36:43.109272+0000","last_became_active":"2024-05-08T06:36:40.644640+0000","last_became_peered":"2024-05-08T06:36:40.644640+0000","last_unstale":"2024-05-08T06:36:43.109272+0000","last_undegraded":"2024-05-08T06:36:43.109272+0000","last_fullsized":"2024-05-08T06:36:43.109272+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649183+0000","last_change":"2024-05-08T06:36:41.656744+0000","last_active":"2024-05-08T06:36:42.649183+0000","last_peered":"2024-05-08T06:36:42.649183+0000","last_clean":"2024-05-08T06:36:42.649183+0000","last_became_active":"2024-05-08T06:36:41.656483+0000","last_became_peered":"2024-05-08T06:36:41.656483+0000","last_unstale":"2024-05-08T06:36:42.649183+0000","last_undegraded":"2024-05-08T06:36:42.649183+0000","last_fullsized":"2024-05-08T06:36:42.649183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653768+0000","last_change":"2024-05-08T06:36:40.648551+0000","last_active":"2024-05-08T06:36:42.653768+0000","last_peered":"2024-05-08T06:36:42.653768+0000","last_clean":"2024-05-08T06:36:42.653768+0000","last_became_active":"2024-05-08T06:36:40.648265+0000","last_became_peered":"2024-05-08T06:36:40.648265+0000","last_unstale":"2024-05-08T06:36:42.653768+0000","last_undegraded":"2024-05-08T06:36:42.653768+0000","last_fullsized":"2024-05-08T06:36:42.653768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109024+0000","last_change":"2024-05-08T06:36:41.658637+0000","last_active":"2024-05-08T06:36:43.109024+0000","last_peered":"2024-05-08T06:36:43.109024+0000","last_clean":"2024-05-08T06:36:43.109024+0000","last_became_active":"2024-05-08T06:36:41.658333+0000","last_became_peered":"2024-05-08T06:36:41.658333+0000","last_unstale":"2024-05-08T06:36:43.109024+0000","last_undegraded":"2024-05-08T06:36:43.109024+0000","last_fullsized":"2024-05-08T06:36:43.109024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109045+0000","last_change":"2024-05-08T06:36:40.647645+0000","last_active":"2024-05-08T06:36:43.109045+0000","last_peered":"2024-05-08T06:36:43.109045+0000","last_clean":"2024-05-08T06:36:43.109045+0000","last_became_active":"2024-05-08T06:36:40.647368+0000","last_became_peered":"2024-05-08T06:36:40.647368+0000","last_unstale":"2024-05-08T06:36:43.109045+0000","last_undegraded":"2024-05-08T06:36:43.109045+0000","last_fullsized":"2024-05-08T06:36:43.109045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109122+0000","last_change":"2024-05-08T06:36:41.662012+0000","last_active":"2024-05-08T06:36:43.109122+0000","last_peered":"2024-05-08T06:36:43.109122+0000","last_clean":"2024-05-08T06:36:43.109122+0000","last_became_active":"2024-05-08T06:36:41.661780+0000","last_became_peered":"2024-05-08T06:36:41.661780+0000","last_unstale":"2024-05-08T06:36:43.109122+0000","last_undegraded":"2024-05-08T06:36:43.109122+0000","last_fullsized":"2024-05-08T06:36:43.109122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649372+0000","last_change":"2024-05-08T06:36:40.649405+0000","last_active":"2024-05-08T06:36:42.649372+0000","last_peered":"2024-05-08T06:36:42.649372+0000","last_clean":"2024-05-08T06:36:42.649372+0000","last_became_active":"2024-05-08T06:36:40.649172+0000","last_became_peered":"2024-05-08T06:36:40.649172+0000","last_unstale":"2024-05-08T06:36:42.649372+0000","last_undegraded":"2024-05-08T06:36:42.649372+0000","last_fullsized":"2024-05-08T06:36:42.649372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654679+0000","last_change":"2024-05-08T06:36:41.654141+0000","last_active":"2024-05-08T06:36:42.654679+0000","last_peered":"2024-05-08T06:36:42.654679+0000","last_clean":"2024-05-08T06:36:42.654679+0000","last_became_active":"2024-05-08T06:36:41.653723+0000","last_became_peered":"2024-05-08T06:36:41.653723+0000","last_unstale":"2024-05-08T06:36:42.654679+0000","last_undegraded":"2024-05-08T06:36:42.654679+0000","last_fullsized":"2024-05-08T06:36:42.654679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:04.579668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654000+0000","last_change":"2024-05-08T06:36:40.641783+0000","last_active":"2024-05-08T06:36:42.654000+0000","last_peered":"2024-05-08T06:36:42.654000+0000","last_clean":"2024-05-08T06:36:42.654000+0000","last_became_active":"2024-05-08T06:36:40.641270+0000","last_became_peered":"2024-05-08T06:36:40.641270+0000","last_unstale":"2024-05-08T06:36:42.654000+0000","last_undegraded":"2024-05-08T06:36:42.654000+0000","last_fullsized":"2024-05-08T06:36:42.654000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108979+0000","last_change":"2024-05-08T06:36:41.657579+0000","last_active":"2024-05-08T06:36:43.108979+0000","last_peered":"2024-05-08T06:36:43.108979+0000","last_clean":"2024-05-08T06:36:43.108979+0000","last_became_active":"2024-05-08T06:36:41.657299+0000","last_became_peered":"2024-05-08T06:36:41.657299+0000","last_unstale":"2024-05-08T06:36:43.108979+0000","last_undegraded":"2024-05-08T06:36:43.108979+0000","last_fullsized":"2024-05-08T06:36:43.108979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:02.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108685+0000","last_change":"2024-05-08T06:36:40.641651+0000","last_active":"2024-05-08T06:36:43.108685+0000","last_peered":"2024-05-08T06:36:43.108685+0000","last_clean":"2024-05-08T06:36:43.108685+0000","last_became_active":"2024-05-08T06:36:40.641168+0000","last_became_peered":"2024-05-08T06:36:40.641168+0000","last_unstale":"2024-05-08T06:36:43.108685+0000","last_undegraded":"2024-05-08T06:36:43.108685+0000","last_fullsized":"2024-05-08T06:36:43.108685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:47.997246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108903+0000","last_change":"2024-05-08T06:36:41.661356+0000","last_active":"2024-05-08T06:36:43.108903+0000","last_peered":"2024-05-08T06:36:43.108903+0000","last_clean":"2024-05-08T06:36:43.108903+0000","last_became_active":"2024-05-08T06:36:41.661085+0000","last_became_peered":"2024-05-08T06:36:41.661085+0000","last_unstale":"2024-05-08T06:36:43.108903+0000","last_undegraded":"2024-05-08T06:36:43.108903+0000","last_fullsized":"2024-05-08T06:36:43.108903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109770+0000","last_change":"2024-05-08T06:36:40.646694+0000","last_active":"2024-05-08T06:36:43.109770+0000","last_peered":"2024-05-08T06:36:43.109770+0000","last_clean":"2024-05-08T06:36:43.109770+0000","last_became_active":"2024-05-08T06:36:40.645833+0000","last_became_peered":"2024-05-08T06:36:40.645833+0000","last_unstale":"2024-05-08T06:36:43.109770+0000","last_undegraded":"2024-05-08T06:36:43.109770+0000","last_fullsized":"2024-05-08T06:36:43.109770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109418+0000","last_change":"2024-05-08T06:36:41.668787+0000","last_active":"2024-05-08T06:36:43.109418+0000","last_peered":"2024-05-08T06:36:43.109418+0000","last_clean":"2024-05-08T06:36:43.109418+0000","last_became_active":"2024-05-08T06:36:41.668489+0000","last_became_peered":"2024-05-08T06:36:41.668489+0000","last_unstale":"2024-05-08T06:36:43.109418+0000","last_undegraded":"2024-05-08T06:36:43.109418+0000","last_fullsized":"2024-05-08T06:36:43.109418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.395237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653600+0000","last_change":"2024-05-08T06:36:40.640082+0000","last_active":"2024-05-08T06:36:42.653600+0000","last_peered":"2024-05-08T06:36:42.653600+0000","last_clean":"2024-05-08T06:36:42.653600+0000","last_became_active":"2024-05-08T06:36:40.638558+0000","last_became_peered":"2024-05-08T06:36:40.638558+0000","last_unstale":"2024-05-08T06:36:42.653600+0000","last_undegraded":"2024-05-08T06:36:42.653600+0000","last_fullsized":"2024-05-08T06:36:42.653600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653523+0000","last_change":"2024-05-08T06:36:41.643855+0000","last_active":"2024-05-08T06:36:42.653523+0000","last_peered":"2024-05-08T06:36:42.653523+0000","last_clean":"2024-05-08T06:36:42.653523+0000","last_became_active":"2024-05-08T06:36:41.643117+0000","last_became_peered":"2024-05-08T06:36:41.643117+0000","last_unstale":"2024-05-08T06:36:42.653523+0000","last_undegraded":"2024-05-08T06:36:42.653523+0000","last_fullsized":"2024-05-08T06:36:42.653523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110713+0000","last_change":"2024-05-08T06:36:40.642303+0000","last_active":"2024-05-08T06:36:43.110713+0000","last_peered":"2024-05-08T06:36:43.110713+0000","last_clean":"2024-05-08T06:36:43.110713+0000","last_became_active":"2024-05-08T06:36:40.642042+0000","last_became_peered":"2024-05-08T06:36:40.642042+0000","last_unstale":"2024-05-08T06:36:43.110713+0000","last_undegraded":"2024-05-08T06:36:43.110713+0000","last_fullsized":"2024-05-08T06:36:43.110713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109058+0000","last_change":"2024-05-08T06:36:41.660559+0000","last_active":"2024-05-08T06:36:43.109058+0000","last_peered":"2024-05-08T06:36:43.109058+0000","last_clean":"2024-05-08T06:36:43.109058+0000","last_became_active":"2024-05-08T06:36:41.660282+0000","last_became_peered":"2024-05-08T06:36:41.660282+0000","last_unstale":"2024-05-08T06:36:43.109058+0000","last_undegraded":"2024-05-08T06:36:43.109058+0000","last_fullsized":"2024-05-08T06:36:43.109058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:49.740862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654374+0000","last_change":"2024-05-08T06:36:40.646655+0000","last_active":"2024-05-08T06:36:42.654374+0000","last_peered":"2024-05-08T06:36:42.654374+0000","last_clean":"2024-05-08T06:36:42.654374+0000","last_became_active":"2024-05-08T06:36:40.646346+0000","last_became_peered":"2024-05-08T06:36:40.646346+0000","last_unstale":"2024-05-08T06:36:42.654374+0000","last_undegraded":"2024-05-08T06:36:42.654374+0000","last_fullsized":"2024-05-08T06:36:42.654374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109890+0000","last_change":"2024-05-08T06:36:41.659817+0000","last_active":"2024-05-08T06:36:43.109890+0000","last_peered":"2024-05-08T06:36:43.109890+0000","last_clean":"2024-05-08T06:36:43.109890+0000","last_became_active":"2024-05-08T06:36:41.659524+0000","last_became_peered":"2024-05-08T06:36:41.659524+0000","last_unstale":"2024-05-08T06:36:43.109890+0000","last_undegraded":"2024-05-08T06:36:43.109890+0000","last_fullsized":"2024-05-08T06:36:43.109890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:38:12.052561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110881+0000","last_change":"2024-05-08T06:36:40.641491+0000","last_active":"2024-05-08T06:36:43.110881+0000","last_peered":"2024-05-08T06:36:43.110881+0000","last_clean":"2024-05-08T06:36:43.110881+0000","last_became_active":"2024-05-08T06:36:40.641207+0000","last_became_peered":"2024-05-08T06:36:40.641207+0000","last_unstale":"2024-05-08T06:36:43.110881+0000","last_undegraded":"2024-05-08T06:36:43.110881+0000","last_fullsized":"2024-05-08T06:36:43.110881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110049+0000","last_change":"2024-05-08T06:36:41.667992+0000","last_active":"2024-05-08T06:36:43.110049+0000","last_peered":"2024-05-08T06:36:43.110049+0000","last_clean":"2024-05-08T06:36:43.110049+0000","last_became_active":"2024-05-08T06:36:41.667733+0000","last_became_peered":"2024-05-08T06:36:41.667733+0000","last_unstale":"2024-05-08T06:36:43.110049+0000","last_undegraded":"2024-05-08T06:36:43.110049+0000","last_fullsized":"2024-05-08T06:36:43.110049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:41:00.520597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649448+0000","last_change":"2024-05-08T06:36:40.646069+0000","last_active":"2024-05-08T06:36:42.649448+0000","last_peered":"2024-05-08T06:36:42.649448+0000","last_clean":"2024-05-08T06:36:42.649448+0000","last_became_active":"2024-05-08T06:36:40.645769+0000","last_became_peered":"2024-05-08T06:36:40.645769+0000","last_unstale":"2024-05-08T06:36:42.649448+0000","last_undegraded":"2024-05-08T06:36:42.649448+0000","last_fullsized":"2024-05-08T06:36:42.649448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:25.837342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109628+0000","last_change":"2024-05-08T06:36:41.660346+0000","last_active":"2024-05-08T06:36:43.109628+0000","last_peered":"2024-05-08T06:36:43.109628+0000","last_clean":"2024-05-08T06:36:43.109628+0000","last_became_active":"2024-05-08T06:36:41.660028+0000","last_became_peered":"2024-05-08T06:36:41.660028+0000","last_unstale":"2024-05-08T06:36:43.109628+0000","last_undegraded":"2024-05-08T06:36:43.109628+0000","last_fullsized":"2024-05-08T06:36:43.109628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:14.886723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110847+0000","last_change":"2024-05-08T06:36:40.647112+0000","last_active":"2024-05-08T06:36:43.110847+0000","last_peered":"2024-05-08T06:36:43.110847+0000","last_clean":"2024-05-08T06:36:43.110847+0000","last_became_active":"2024-05-08T06:36:40.646804+0000","last_became_peered":"2024-05-08T06:36:40.646804+0000","last_unstale":"2024-05-08T06:36:43.110847+0000","last_undegraded":"2024-05-08T06:36:43.110847+0000","last_fullsized":"2024-05-08T06:36:43.110847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109148+0000","last_change":"2024-05-08T06:36:41.663875+0000","last_active":"2024-05-08T06:36:43.109148+0000","last_peered":"2024-05-08T06:36:43.109148+0000","last_clean":"2024-05-08T06:36:43.109148+0000","last_became_active":"2024-05-08T06:36:41.663595+0000","last_became_peered":"2024-05-08T06:36:41.663595+0000","last_unstale":"2024-05-08T06:36:43.109148+0000","last_undegraded":"2024-05-08T06:36:43.109148+0000","last_fullsized":"2024-05-08T06:36:43.109148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:42.444444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108725+0000","last_change":"2024-05-08T06:36:31.710287+0000","last_active":"2024-05-08T06:36:43.108725+0000","last_peered":"2024-05-08T06:36:43.108725+0000","last_clean":"2024-05-08T06:36:43.108725+0000","last_became_active":"2024-05-08T06:36:31.709270+0000","last_became_peered":"2024-05-08T06:36:31.709270+0000","last_unstale":"2024-05-08T06:36:43.108725+0000","last_undegraded":"2024-05-08T06:36:43.108725+0000","last_fullsized":"2024-05-08T06:36:43.108725+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_clean_scrub_stamp":"2024-05-08T06:36:30.122918+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:42.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108783+0000","last_change":"2024-05-08T06:36:40.645536+0000","last_active":"2024-05-08T06:36:43.108783+0000","last_peered":"2024-05-08T06:36:43.108783+0000","last_clean":"2024-05-08T06:36:43.108783+0000","last_became_active":"2024-05-08T06:36:40.644256+0000","last_became_peered":"2024-05-08T06:36:40.644256+0000","last_unstale":"2024-05-08T06:36:43.108783+0000","last_undegraded":"2024-05-08T06:36:43.108783+0000","last_fullsized":"2024-05-08T06:36:43.108783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109026+0000","last_change":"2024-05-08T06:36:41.655301+0000","last_active":"2024-05-08T06:36:43.109026+0000","last_peered":"2024-05-08T06:36:43.109026+0000","last_clean":"2024-05-08T06:36:43.109026+0000","last_became_active":"2024-05-08T06:36:41.655090+0000","last_became_peered":"2024-05-08T06:36:41.655090+0000","last_unstale":"2024-05-08T06:36:43.109026+0000","last_undegraded":"2024-05-08T06:36:43.109026+0000","last_fullsized":"2024-05-08T06:36:43.109026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108653+0000","last_change":"2024-05-08T06:36:40.647913+0000","last_active":"2024-05-08T06:36:43.108653+0000","last_peered":"2024-05-08T06:36:43.108653+0000","last_clean":"2024-05-08T06:36:43.108653+0000","last_became_active":"2024-05-08T06:36:40.647638+0000","last_became_peered":"2024-05-08T06:36:40.647638+0000","last_unstale":"2024-05-08T06:36:43.108653+0000","last_undegraded":"2024-05-08T06:36:43.108653+0000","last_fullsized":"2024-05-08T06:36:43.108653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:29:44.646218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108285+0000","last_change":"2024-05-08T06:36:41.658563+0000","last_active":"2024-05-08T06:36:43.108285+0000","last_peered":"2024-05-08T06:36:43.108285+0000","last_clean":"2024-05-08T06:36:43.108285+0000","last_became_active":"2024-05-08T06:36:41.657157+0000","last_became_peered":"2024-05-08T06:36:41.657157+0000","last_unstale":"2024-05-08T06:36:43.108285+0000","last_undegraded":"2024-05-08T06:36:43.108285+0000","last_fullsized":"2024-05-08T06:36:43.108285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:30:14.798179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110812+0000","last_change":"2024-05-08T06:36:40.642187+0000","last_active":"2024-05-08T06:36:43.110812+0000","last_peered":"2024-05-08T06:36:43.110812+0000","last_clean":"2024-05-08T06:36:43.110812+0000","last_became_active":"2024-05-08T06:36:40.641906+0000","last_became_peered":"2024-05-08T06:36:40.641906+0000","last_unstale":"2024-05-08T06:36:43.110812+0000","last_undegraded":"2024-05-08T06:36:43.110812+0000","last_fullsized":"2024-05-08T06:36:43.110812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649176+0000","last_change":"2024-05-08T06:36:41.642328+0000","last_active":"2024-05-08T06:36:42.649176+0000","last_peered":"2024-05-08T06:36:42.649176+0000","last_clean":"2024-05-08T06:36:42.649176+0000","last_became_active":"2024-05-08T06:36:41.642053+0000","last_became_peered":"2024-05-08T06:36:41.642053+0000","last_unstale":"2024-05-08T06:36:42.649176+0000","last_undegraded":"2024-05-08T06:36:42.649176+0000","last_fullsized":"2024-05-08T06:36:42.649176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109887+0000","last_change":"2024-05-08T06:36:40.646469+0000","last_active":"2024-05-08T06:36:43.109887+0000","last_peered":"2024-05-08T06:36:43.109887+0000","last_clean":"2024-05-08T06:36:43.109887+0000","last_became_active":"2024-05-08T06:36:40.646207+0000","last_became_peered":"2024-05-08T06:36:40.646207+0000","last_unstale":"2024-05-08T06:36:43.109887+0000","last_undegraded":"2024-05-08T06:36:43.109887+0000","last_fullsized":"2024-05-08T06:36:43.109887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654706+0000","last_change":"2024-05-08T06:36:41.654729+0000","last_active":"2024-05-08T06:36:42.654706+0000","last_peered":"2024-05-08T06:36:42.654706+0000","last_clean":"2024-05-08T06:36:42.654706+0000","last_became_active":"2024-05-08T06:36:41.654466+0000","last_became_peered":"2024-05-08T06:36:41.654466+0000","last_unstale":"2024-05-08T06:36:42.654706+0000","last_undegraded":"2024-05-08T06:36:42.654706+0000","last_fullsized":"2024-05-08T06:36:42.654706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:44.798191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649223+0000","last_change":"2024-05-08T06:36:40.647304+0000","last_active":"2024-05-08T06:36:42.649223+0000","last_peered":"2024-05-08T06:36:42.649223+0000","last_clean":"2024-05-08T06:36:42.649223+0000","last_became_active":"2024-05-08T06:36:40.647010+0000","last_became_peered":"2024-05-08T06:36:40.647010+0000","last_unstale":"2024-05-08T06:36:42.649223+0000","last_undegraded":"2024-05-08T06:36:42.649223+0000","last_fullsized":"2024-05-08T06:36:42.649223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:50.353194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109655+0000","last_change":"2024-05-08T06:36:41.656064+0000","last_active":"2024-05-08T06:36:43.109655+0000","last_peered":"2024-05-08T06:36:43.109655+0000","last_clean":"2024-05-08T06:36:43.109655+0000","last_became_active":"2024-05-08T06:36:41.655714+0000","last_became_peered":"2024-05-08T06:36:41.655714+0000","last_unstale":"2024-05-08T06:36:43.109655+0000","last_undegraded":"2024-05-08T06:36:43.109655+0000","last_fullsized":"2024-05-08T06:36:43.109655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.395237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109737+0000","last_change":"2024-05-08T06:36:40.648662+0000","last_active":"2024-05-08T06:36:43.109737+0000","last_peered":"2024-05-08T06:36:43.109737+0000","last_clean":"2024-05-08T06:36:43.109737+0000","last_became_active":"2024-05-08T06:36:40.648367+0000","last_became_peered":"2024-05-08T06:36:40.648367+0000","last_unstale":"2024-05-08T06:36:43.109737+0000","last_undegraded":"2024-05-08T06:36:43.109737+0000","last_fullsized":"2024-05-08T06:36:43.109737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:25.837342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649254+0000","last_change":"2024-05-08T06:36:41.642330+0000","last_active":"2024-05-08T06:36:42.649254+0000","last_peered":"2024-05-08T06:36:42.649254+0000","last_clean":"2024-05-08T06:36:42.649254+0000","last_became_active":"2024-05-08T06:36:41.642059+0000","last_became_peered":"2024-05-08T06:36:41.642059+0000","last_unstale":"2024-05-08T06:36:42.649254+0000","last_undegraded":"2024-05-08T06:36:42.649254+0000","last_fullsized":"2024-05-08T06:36:42.649254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649350+0000","last_change":"2024-05-08T06:36:40.646675+0000","last_active":"2024-05-08T06:36:42.649350+0000","last_peered":"2024-05-08T06:36:42.649350+0000","last_clean":"2024-05-08T06:36:42.649350+0000","last_became_active":"2024-05-08T06:36:40.646346+0000","last_became_peered":"2024-05-08T06:36:40.646346+0000","last_unstale":"2024-05-08T06:36:42.649350+0000","last_undegraded":"2024-05-08T06:36:42.649350+0000","last_fullsized":"2024-05-08T06:36:42.649350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:03.574076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109189+0000","last_change":"2024-05-08T06:36:41.658633+0000","last_active":"2024-05-08T06:36:43.109189+0000","last_peered":"2024-05-08T06:36:43.109189+0000","last_clean":"2024-05-08T06:36:43.109189+0000","last_became_active":"2024-05-08T06:36:41.658294+0000","last_became_peered":"2024-05-08T06:36:41.658294+0000","last_unstale":"2024-05-08T06:36:43.109189+0000","last_undegraded":"2024-05-08T06:36:43.109189+0000","last_fullsized":"2024-05-08T06:36:43.109189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653882+0000","last_change":"2024-05-08T06:36:40.639547+0000","last_active":"2024-05-08T06:36:42.653882+0000","last_peered":"2024-05-08T06:36:42.653882+0000","last_clean":"2024-05-08T06:36:42.653882+0000","last_became_active":"2024-05-08T06:36:40.639240+0000","last_became_peered":"2024-05-08T06:36:40.639240+0000","last_unstale":"2024-05-08T06:36:42.653882+0000","last_undegraded":"2024-05-08T06:36:42.653882+0000","last_fullsized":"2024-05-08T06:36:42.653882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108530+0000","last_change":"2024-05-08T06:36:41.648709+0000","last_active":"2024-05-08T06:36:43.108530+0000","last_peered":"2024-05-08T06:36:43.108530+0000","last_clean":"2024-05-08T06:36:43.108530+0000","last_became_active":"2024-05-08T06:36:41.648406+0000","last_became_peered":"2024-05-08T06:36:41.648406+0000","last_unstale":"2024-05-08T06:36:43.108530+0000","last_undegraded":"2024-05-08T06:36:43.108530+0000","last_fullsized":"2024-05-08T06:36:43.108530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108653+0000","last_change":"2024-05-08T06:36:40.648721+0000","last_active":"2024-05-08T06:36:43.108653+0000","last_peered":"2024-05-08T06:36:43.108653+0000","last_clean":"2024-05-08T06:36:43.108653+0000","last_became_active":"2024-05-08T06:36:40.648429+0000","last_became_peered":"2024-05-08T06:36:40.648429+0000","last_unstale":"2024-05-08T06:36:43.108653+0000","last_undegraded":"2024-05-08T06:36:43.108653+0000","last_fullsized":"2024-05-08T06:36:43.108653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:43.792599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654290+0000","last_change":"2024-05-08T06:36:41.655289+0000","last_active":"2024-05-08T06:36:42.654290+0000","last_peered":"2024-05-08T06:36:42.654290+0000","last_clean":"2024-05-08T06:36:42.654290+0000","last_became_active":"2024-05-08T06:36:41.655022+0000","last_became_peered":"2024-05-08T06:36:41.655022+0000","last_unstale":"2024-05-08T06:36:42.654290+0000","last_undegraded":"2024-05-08T06:36:42.654290+0000","last_fullsized":"2024-05-08T06:36:42.654290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:57.405934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649256+0000","last_change":"2024-05-08T06:36:40.645335+0000","last_active":"2024-05-08T06:36:42.649256+0000","last_peered":"2024-05-08T06:36:42.649256+0000","last_clean":"2024-05-08T06:36:42.649256+0000","last_became_active":"2024-05-08T06:36:40.645019+0000","last_became_peered":"2024-05-08T06:36:40.645019+0000","last_unstale":"2024-05-08T06:36:42.649256+0000","last_undegraded":"2024-05-08T06:36:42.649256+0000","last_fullsized":"2024-05-08T06:36:42.649256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108851+0000","last_change":"2024-05-08T06:36:41.653910+0000","last_active":"2024-05-08T06:36:43.108851+0000","last_peered":"2024-05-08T06:36:43.108851+0000","last_clean":"2024-05-08T06:36:43.108851+0000","last_became_active":"2024-05-08T06:36:41.653567+0000","last_became_peered":"2024-05-08T06:36:41.653567+0000","last_unstale":"2024-05-08T06:36:43.108851+0000","last_undegraded":"2024-05-08T06:36:43.108851+0000","last_fullsized":"2024-05-08T06:36:43.108851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649426+0000","last_change":"2024-05-08T06:36:40.642880+0000","last_active":"2024-05-08T06:36:42.649426+0000","last_peered":"2024-05-08T06:36:42.649426+0000","last_clean":"2024-05-08T06:36:42.649426+0000","last_became_active":"2024-05-08T06:36:40.642605+0000","last_became_peered":"2024-05-08T06:36:40.642605+0000","last_unstale":"2024-05-08T06:36:42.649426+0000","last_undegraded":"2024-05-08T06:36:42.649426+0000","last_fullsized":"2024-05-08T06:36:42.649426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109118+0000","last_change":"2024-05-08T06:36:41.654124+0000","last_active":"2024-05-08T06:36:43.109118+0000","last_peered":"2024-05-08T06:36:43.109118+0000","last_clean":"2024-05-08T06:36:43.109118+0000","last_became_active":"2024-05-08T06:36:41.653837+0000","last_became_peered":"2024-05-08T06:36:41.653837+0000","last_unstale":"2024-05-08T06:36:43.109118+0000","last_undegraded":"2024-05-08T06:36:43.109118+0000","last_fullsized":"2024-05-08T06:36:43.109118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109611+0000","last_change":"2024-05-08T06:36:40.644938+0000","last_active":"2024-05-08T06:36:43.109611+0000","last_peered":"2024-05-08T06:36:43.109611+0000","last_clean":"2024-05-08T06:36:43.109611+0000","last_became_active":"2024-05-08T06:36:40.644549+0000","last_became_peered":"2024-05-08T06:36:40.644549+0000","last_unstale":"2024-05-08T06:36:43.109611+0000","last_undegraded":"2024-05-08T06:36:43.109611+0000","last_fullsized":"2024-05-08T06:36:43.109611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:50.353194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653954+0000","last_change":"2024-05-08T06:36:41.659401+0000","last_active":"2024-05-08T06:36:42.653954+0000","last_peered":"2024-05-08T06:36:42.653954+0000","last_clean":"2024-05-08T06:36:42.653954+0000","last_became_active":"2024-05-08T06:36:41.659063+0000","last_became_peered":"2024-05-08T06:36:41.659063+0000","last_unstale":"2024-05-08T06:36:42.653954+0000","last_undegraded":"2024-05-08T06:36:42.653954+0000","last_fullsized":"2024-05-08T06:36:42.653954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109449+0000","last_change":"2024-05-08T06:36:41.655858+0000","last_active":"2024-05-08T06:36:43.109449+0000","last_peered":"2024-05-08T06:36:43.109449+0000","last_clean":"2024-05-08T06:36:43.109449+0000","last_became_active":"2024-05-08T06:36:41.655569+0000","last_became_peered":"2024-05-08T06:36:41.655569+0000","last_unstale":"2024-05-08T06:36:43.109449+0000","last_undegraded":"2024-05-08T06:36:43.109449+0000","last_fullsized":"2024-05-08T06:36:43.109449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:02.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109476+0000","last_change":"2024-05-08T06:36:40.646643+0000","last_active":"2024-05-08T06:36:43.109476+0000","last_peered":"2024-05-08T06:36:43.109476+0000","last_clean":"2024-05-08T06:36:43.109476+0000","last_became_active":"2024-05-08T06:36:40.646319+0000","last_became_peered":"2024-05-08T06:36:40.646319+0000","last_unstale":"2024-05-08T06:36:43.109476+0000","last_undegraded":"2024-05-08T06:36:43.109476+0000","last_fullsized":"2024-05-08T06:36:43.109476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:03.574076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109441+0000","last_change":"2024-05-08T06:36:41.656383+0000","last_active":"2024-05-08T06:36:43.109441+0000","last_peered":"2024-05-08T06:36:43.109441+0000","last_clean":"2024-05-08T06:36:43.109441+0000","last_became_active":"2024-05-08T06:36:41.656059+0000","last_became_peered":"2024-05-08T06:36:41.656059+0000","last_unstale":"2024-05-08T06:36:43.109441+0000","last_undegraded":"2024-05-08T06:36:43.109441+0000","last_fullsized":"2024-05-08T06:36:43.109441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:49.740862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109265+0000","last_change":"2024-05-08T06:36:40.646560+0000","last_active":"2024-05-08T06:36:43.109265+0000","last_peered":"2024-05-08T06:36:43.109265+0000","last_clean":"2024-05-08T06:36:43.109265+0000","last_became_active":"2024-05-08T06:36:40.646285+0000","last_became_peered":"2024-05-08T06:36:40.646285+0000","last_unstale":"2024-05-08T06:36:43.109265+0000","last_undegraded":"2024-05-08T06:36:43.109265+0000","last_fullsized":"2024-05-08T06:36:43.109265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108439+0000","last_change":"2024-05-08T06:36:41.661975+0000","last_active":"2024-05-08T06:36:43.108439+0000","last_peered":"2024-05-08T06:36:43.108439+0000","last_clean":"2024-05-08T06:36:43.108439+0000","last_became_active":"2024-05-08T06:36:41.661759+0000","last_became_peered":"2024-05-08T06:36:41.661759+0000","last_unstale":"2024-05-08T06:36:43.108439+0000","last_undegraded":"2024-05-08T06:36:43.108439+0000","last_fullsized":"2024-05-08T06:36:43.108439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109215+0000","last_change":"2024-05-08T06:36:40.645426+0000","last_active":"2024-05-08T06:36:43.109215+0000","last_peered":"2024-05-08T06:36:43.109215+0000","last_clean":"2024-05-08T06:36:43.109215+0000","last_became_active":"2024-05-08T06:36:40.645194+0000","last_became_peered":"2024-05-08T06:36:40.645194+0000","last_unstale":"2024-05-08T06:36:43.109215+0000","last_undegraded":"2024-05-08T06:36:43.109215+0000","last_fullsized":"2024-05-08T06:36:43.109215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109153+0000","last_change":"2024-05-08T06:36:41.660148+0000","last_active":"2024-05-08T06:36:43.109153+0000","last_peered":"2024-05-08T06:36:43.109153+0000","last_clean":"2024-05-08T06:36:43.109153+0000","last_became_active":"2024-05-08T06:36:41.659907+0000","last_became_peered":"2024-05-08T06:36:41.659907+0000","last_unstale":"2024-05-08T06:36:43.109153+0000","last_undegraded":"2024-05-08T06:36:43.109153+0000","last_fullsized":"2024-05-08T06:36:43.109153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:45:07.961465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109158+0000","last_change":"2024-05-08T06:36:40.645379+0000","last_active":"2024-05-08T06:36:43.109158+0000","last_peered":"2024-05-08T06:36:43.109158+0000","last_clean":"2024-05-08T06:36:43.109158+0000","last_became_active":"2024-05-08T06:36:40.645091+0000","last_became_peered":"2024-05-08T06:36:40.645091+0000","last_unstale":"2024-05-08T06:36:43.109158+0000","last_undegraded":"2024-05-08T06:36:43.109158+0000","last_fullsized":"2024-05-08T06:36:43.109158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:36:50.067 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:36:50.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.181+0000 7efc075c1640 1 Processor -- start 2024-05-08T06:36:50.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.181+0000 7efc075c1640 1 -- start start 2024-05-08T06:36:50.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.182+0000 7efc075c1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc0015f640 0x7efc000d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.182+0000 7efc075c1640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efc000b3f50 0x7efc000b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.182+0000 7efc075c1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc000d1240 con 0x7efc000d3940 2024-05-08T06:36:50.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.182+0000 7efc075c1640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7efc000d3750 con 0x7efc000b3f50 2024-05-08T06:36:50.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.182+0000 7efc075c1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efc000d2750 con 0x7efc0015f640 2024-05-08T06:36:50.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.182+0000 7efc05b5e640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7efc000d3940 0x7efc000b1ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57018/0 (socket says 172.21.15.163:57018) 2024-05-08T06:36:50.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.182+0000 7efc05b5e640 1 -- 172.21.15.163:0/1425400787 learned_addr learned my addr 172.21.15.163:0/1425400787 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:50.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.182+0000 7efc04b5c640 1 --2- 172.21.15.163:0/1425400787 >> v2:172.21.15.192:3300/0 conn(0x7efc000b3f50 0x7efc000b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.182+0000 7efc0535d640 1 --2- 172.21.15.163:0/1425400787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc0015f640 0x7efc000d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.182+0000 7efbfffff640 1 -- 172.21.15.163:0/1425400787 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3948801349 0 0) 0x7efc000d1240 con 0x7efc000d3940 2024-05-08T06:36:50.183 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.183+0000 7efbfffff640 1 -- 172.21.15.163:0/1425400787 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efbe8003520 con 0x7efc000d3940 2024-05-08T06:36:50.183 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.183+0000 7efc04b5c640 1 -- 172.21.15.163:0/1425400787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc0015f640 msgr2=0x7efc000d3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:50.183 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.183+0000 7efc04b5c640 1 --2- 172.21.15.163:0/1425400787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc0015f640 0x7efc000d3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.183 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.183+0000 7efc04b5c640 1 -- 172.21.15.163:0/1425400787 >> v1:172.21.15.163:6789/0 conn(0x7efc000d3940 legacy=0x7efc000b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.183 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.183+0000 7efc04b5c640 1 -- 172.21.15.163:0/1425400787 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc000d5c10 con 0x7efc000b3f50 2024-05-08T06:36:50.183 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.183+0000 7efc0535d640 1 --2- 172.21.15.163:0/1425400787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc0015f640 0x7efc000d3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:50.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.183+0000 7efc04b5c640 1 --2- 172.21.15.163:0/1425400787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc000b3f50 0x7efc000b43d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009d90 tx=0x7efbf402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39cb78e153ad1444 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:50.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.184+0000 7efbfffff640 1 -- 172.21.15.163:0/1425400787 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4007740 con 0x7efc000b3f50 2024-05-08T06:36:50.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.184+0000 7efbfffff640 1 -- 172.21.15.163:0/1425400787 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbf40078e0 con 0x7efc000b3f50 2024-05-08T06:36:50.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.184+0000 7efbfffff640 1 -- 172.21.15.163:0/1425400787 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4007c90 con 0x7efc000b3f50 2024-05-08T06:36:50.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.184+0000 7efc075c1640 1 -- 172.21.15.163:0/1425400787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc000b3f50 msgr2=0x7efc000b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.184+0000 7efc075c1640 1 --2- 172.21.15.163:0/1425400787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc000b3f50 0x7efc000b43d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009d90 tx=0x7efbf402a660 comp rx=0 tx=0).stop 2024-05-08T06:36:50.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.184+0000 7efc075c1640 1 -- 172.21.15.163:0/1425400787 shutdown_connections 2024-05-08T06:36:50.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.184+0000 7efc075c1640 1 --2- 172.21.15.163:0/1425400787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc000b3f50 0x7efc000b43d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.184+0000 7efc075c1640 1 --2- 172.21.15.163:0/1425400787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc0015f640 0x7efc000d3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.184+0000 7efc075c1640 1 -- 172.21.15.163:0/1425400787 >> 172.21.15.163:0/1425400787 conn(0x7efc0009e5b0 msgr2=0x7efc000d5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:50.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.184+0000 7efc075c1640 1 -- 172.21.15.163:0/1425400787 shutdown_connections 2024-05-08T06:36:50.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.184+0000 7efc075c1640 1 -- 172.21.15.163:0/1425400787 wait complete. 2024-05-08T06:36:50.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.186+0000 7efc075c1640 1 Processor -- start 2024-05-08T06:36:50.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.186+0000 7efc075c1640 1 -- start start 2024-05-08T06:36:50.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.186+0000 7efc075c1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc000b3f50 0x7efc0014e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.186+0000 7efc075c1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc000d3940 0x7efc001574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.186+0000 7efc075c1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc0015f640 0x7efc00157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.186+0000 7efc075c1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efc0015fa20 con 0x7efc0015f640 2024-05-08T06:36:50.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.186+0000 7efc075c1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efc000d61d0 con 0x7efc000d3940 2024-05-08T06:36:50.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.186+0000 7efc075c1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efc000d3dd0 con 0x7efc000b3f50 2024-05-08T06:36:50.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.186+0000 7efc05b5e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc0015f640 0x7efc00157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.186+0000 7efc05b5e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc0015f640 0x7efc00157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44328/0 (socket says 172.21.15.163:44328) 2024-05-08T06:36:50.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.186+0000 7efc05b5e640 1 -- 172.21.15.163:0/3021462862 learned_addr learned my addr 172.21.15.163:0/3021462862 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:50.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.187+0000 7efc04b5c640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc000d3940 0x7efc001574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.187+0000 7efc0535d640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc000b3f50 0x7efc0014e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.187+0000 7efc05b5e640 1 -- 172.21.15.163:0/3021462862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc000b3f50 msgr2=0x7efc0014e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.187+0000 7efc05b5e640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc000b3f50 0x7efc0014e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.187+0000 7efc05b5e640 1 -- 172.21.15.163:0/3021462862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc000d3940 msgr2=0x7efc001574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.187+0000 7efc05b5e640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc000d3940 0x7efc001574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.187+0000 7efc05b5e640 1 -- 172.21.15.163:0/3021462862 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0014edb0 con 0x7efc0015f640 2024-05-08T06:36:50.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.187+0000 7efc04b5c640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc000d3940 0x7efc001574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:50.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.187+0000 7efc05b5e640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc0015f640 0x7efc00157a00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efbf800c430 tx=0x7efbf800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:50.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.187+0000 7efbfdffb640 1 -- 172.21.15.163:0/3021462862 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8017070 con 0x7efc0015f640 2024-05-08T06:36:50.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.187+0000 7efbfdffb640 1 -- 172.21.15.163:0/3021462862 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbf8005ce0 con 0x7efc0015f640 2024-05-08T06:36:50.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.187+0000 7efc075c1640 1 -- 172.21.15.163:0/3021462862 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc000cce40 con 0x7efc0015f640 2024-05-08T06:36:50.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.187+0000 7efbfdffb640 1 -- 172.21.15.163:0/3021462862 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf80072d0 con 0x7efc0015f640 2024-05-08T06:36:50.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.187+0000 7efc075c1640 1 -- 172.21.15.163:0/3021462862 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc000cd400 con 0x7efc0015f640 2024-05-08T06:36:50.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.188+0000 7efc075c1640 1 -- 172.21.15.163:0/3021462862 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc001f4d80 con 0x7efc0015f640 2024-05-08T06:36:50.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.189+0000 7efbfdffb640 1 -- 172.21.15.163:0/3021462862 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efbf801a070 con 0x7efc0015f640 2024-05-08T06:36:50.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.189+0000 7efbfdffb640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7efbec0762d0 0x7efbec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.189+0000 7efbfdffb640 1 -- 172.21.15.163:0/3021462862 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efbf8096990 con 0x7efc0015f640 2024-05-08T06:36:50.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.189+0000 7efc0535d640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7efbec0762d0 0x7efbec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.190+0000 7efc0535d640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7efbec0762d0 0x7efbec078790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009a30 tx=0x7efbf0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:50.192 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.194+0000 7efbfdffb640 1 -- 172.21.15.163:0/3021462862 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efc001f4d80 con 0x7efc0015f640 2024-05-08T06:36:50.370 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.371+0000 7efc075c1640 1 -- 172.21.15.163:0/3021462862 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc0006a550 con 0x7efbec0762d0 2024-05-08T06:36:50.376 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.377+0000 7efbfdffb640 1 -- 172.21.15.163:0/3021462862 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251825 (secure 0 0 0) 0x7efc0006a550 con 0x7efbec0762d0 2024-05-08T06:36:50.379 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:50.380 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2024-05-08T06:36:50.382 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.381+0000 7efc075c1640 1 -- 172.21.15.163:0/3021462862 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7efbec0762d0 msgr2=0x7efbec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.382 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.381+0000 7efc075c1640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7efbec0762d0 0x7efbec078790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009a30 tx=0x7efbf0008040 comp rx=0 tx=0).stop 2024-05-08T06:36:50.382 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.381+0000 7efc075c1640 1 -- 172.21.15.163:0/3021462862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc0015f640 msgr2=0x7efc00157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.382 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.381+0000 7efc075c1640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc0015f640 0x7efc00157a00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efbf800c430 tx=0x7efbf800c8f0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.382 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.381+0000 7efc075c1640 1 -- 172.21.15.163:0/3021462862 shutdown_connections 2024-05-08T06:36:50.382 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.381+0000 7efc075c1640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7efbec0762d0 0x7efbec078790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.381+0000 7efc075c1640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc0015f640 0x7efc00157a00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.381+0000 7efc075c1640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc000d3940 0x7efc001574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.381+0000 7efc075c1640 1 --2- 172.21.15.163:0/3021462862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc000b3f50 0x7efc0014e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.381+0000 7efc075c1640 1 -- 172.21.15.163:0/3021462862 >> 172.21.15.163:0/3021462862 conn(0x7efc0009e5b0 msgr2=0x7efc000d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:50.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.381+0000 7efc075c1640 1 -- 172.21.15.163:0/3021462862 shutdown_connections 2024-05-08T06:36:50.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.381+0000 7efc075c1640 1 -- 172.21.15.163:0/3021462862 wait complete. 2024-05-08T06:36:50.391 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-08T06:36:50.099255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.464635"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109935+0000","last_change":"2024-05-08T06:36:41.664385+0000","last_active":"2024-05-08T06:36:43.109935+0000","last_peered":"2024-05-08T06:36:43.109935+0000","last_clean":"2024-05-08T06:36:43.109935+0000","last_became_active":"2024-05-08T06:36:41.664106+0000","last_became_peered":"2024-05-08T06:36:41.664106+0000","last_unstale":"2024-05-08T06:36:43.109935+0000","last_undegraded":"2024-05-08T06:36:43.109935+0000","last_fullsized":"2024-05-08T06:36:43.109935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:08.774451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649403+0000","last_change":"2024-05-08T06:36:41.647763+0000","last_active":"2024-05-08T06:36:42.649403+0000","last_peered":"2024-05-08T06:36:42.649403+0000","last_clean":"2024-05-08T06:36:42.649403+0000","last_became_active":"2024-05-08T06:36:41.647487+0000","last_became_peered":"2024-05-08T06:36:41.647487+0000","last_unstale":"2024-05-08T06:36:42.649403+0000","last_undegraded":"2024-05-08T06:36:42.649403+0000","last_fullsized":"2024-05-08T06:36:42.649403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108764+0000","last_change":"2024-05-08T06:36:41.649945+0000","last_active":"2024-05-08T06:36:43.108764+0000","last_peered":"2024-05-08T06:36:43.108764+0000","last_clean":"2024-05-08T06:36:43.108764+0000","last_became_active":"2024-05-08T06:36:41.649665+0000","last_became_peered":"2024-05-08T06:36:41.649665+0000","last_unstale":"2024-05-08T06:36:43.108764+0000","last_undegraded":"2024-05-08T06:36:43.108764+0000","last_fullsized":"2024-05-08T06:36:43.108764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109325+0000","last_change":"2024-05-08T06:36:41.656179+0000","last_active":"2024-05-08T06:36:43.109325+0000","last_peered":"2024-05-08T06:36:43.109325+0000","last_clean":"2024-05-08T06:36:43.109325+0000","last_became_active":"2024-05-08T06:36:41.655649+0000","last_became_peered":"2024-05-08T06:36:41.655649+0000","last_unstale":"2024-05-08T06:36:43.109325+0000","last_undegraded":"2024-05-08T06:36:43.109325+0000","last_fullsized":"2024-05-08T06:36:43.109325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:45:07.961465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109228+0000","last_change":"2024-05-08T06:36:41.659928+0000","last_active":"2024-05-08T06:36:43.109228+0000","last_peered":"2024-05-08T06:36:43.109228+0000","last_clean":"2024-05-08T06:36:43.109228+0000","last_became_active":"2024-05-08T06:36:41.659408+0000","last_became_peered":"2024-05-08T06:36:41.659408+0000","last_unstale":"2024-05-08T06:36:43.109228+0000","last_undegraded":"2024-05-08T06:36:43.109228+0000","last_fullsized":"2024-05-08T06:36:43.109228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:14.886723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109064+0000","last_change":"2024-05-08T06:36:41.653658+0000","last_active":"2024-05-08T06:36:43.109064+0000","last_peered":"2024-05-08T06:36:43.109064+0000","last_clean":"2024-05-08T06:36:43.109064+0000","last_became_active":"2024-05-08T06:36:41.653285+0000","last_became_peered":"2024-05-08T06:36:41.653285+0000","last_unstale":"2024-05-08T06:36:43.109064+0000","last_undegraded":"2024-05-08T06:36:43.109064+0000","last_fullsized":"2024-05-08T06:36:43.109064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108884+0000","last_change":"2024-05-08T06:36:41.661193+0000","last_active":"2024-05-08T06:36:43.108884+0000","last_peered":"2024-05-08T06:36:43.108884+0000","last_clean":"2024-05-08T06:36:43.108884+0000","last_became_active":"2024-05-08T06:36:41.660907+0000","last_became_peered":"2024-05-08T06:36:41.660907+0000","last_unstale":"2024-05-08T06:36:43.108884+0000","last_undegraded":"2024-05-08T06:36:43.108884+0000","last_fullsized":"2024-05-08T06:36:43.108884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108493+0000","last_change":"2024-05-08T06:36:41.655350+0000","last_active":"2024-05-08T06:36:43.108493+0000","last_peered":"2024-05-08T06:36:43.108493+0000","last_clean":"2024-05-08T06:36:43.108493+0000","last_became_active":"2024-05-08T06:36:41.655064+0000","last_became_peered":"2024-05-08T06:36:41.655064+0000","last_unstale":"2024-05-08T06:36:43.108493+0000","last_undegraded":"2024-05-08T06:36:43.108493+0000","last_fullsized":"2024-05-08T06:36:43.108493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:11:54.263628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109972+0000","last_change":"2024-05-08T06:36:41.663210+0000","last_active":"2024-05-08T06:36:43.109972+0000","last_peered":"2024-05-08T06:36:43.109972+0000","last_clean":"2024-05-08T06:36:43.109972+0000","last_became_active":"2024-05-08T06:36:41.662954+0000","last_became_peered":"2024-05-08T06:36:41.662954+0000","last_unstale":"2024-05-08T06:36:43.109972+0000","last_undegraded":"2024-05-08T06:36:43.109972+0000","last_fullsized":"2024-05-08T06:36:43.109972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:36:32.652292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654330+0000","last_change":"2024-05-08T06:36:41.662910+0000","last_active":"2024-05-08T06:36:42.654330+0000","last_peered":"2024-05-08T06:36:42.654330+0000","last_clean":"2024-05-08T06:36:42.654330+0000","last_became_active":"2024-05-08T06:36:41.662585+0000","last_became_peered":"2024-05-08T06:36:41.662585+0000","last_unstale":"2024-05-08T06:36:42.654330+0000","last_undegraded":"2024-05-08T06:36:42.654330+0000","last_fullsized":"2024-05-08T06:36:42.654330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653737+0000","last_change":"2024-05-08T06:36:41.663863+0000","last_active":"2024-05-08T06:36:42.653737+0000","last_peered":"2024-05-08T06:36:42.653737+0000","last_clean":"2024-05-08T06:36:42.653737+0000","last_became_active":"2024-05-08T06:36:41.663584+0000","last_became_peered":"2024-05-08T06:36:41.663584+0000","last_unstale":"2024-05-08T06:36:42.653737+0000","last_undegraded":"2024-05-08T06:36:42.653737+0000","last_fullsized":"2024-05-08T06:36:42.653737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654617+0000","last_change":"2024-05-08T06:36:41.661979+0000","last_active":"2024-05-08T06:36:42.654617+0000","last_peered":"2024-05-08T06:36:42.654617+0000","last_clean":"2024-05-08T06:36:42.654617+0000","last_became_active":"2024-05-08T06:36:41.660222+0000","last_became_peered":"2024-05-08T06:36:41.660222+0000","last_unstale":"2024-05-08T06:36:42.654617+0000","last_undegraded":"2024-05-08T06:36:42.654617+0000","last_fullsized":"2024-05-08T06:36:42.654617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:26.842934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108871+0000","last_change":"2024-05-08T06:36:41.654832+0000","last_active":"2024-05-08T06:36:43.108871+0000","last_peered":"2024-05-08T06:36:43.108871+0000","last_clean":"2024-05-08T06:36:43.108871+0000","last_became_active":"2024-05-08T06:36:41.654550+0000","last_became_peered":"2024-05-08T06:36:41.654550+0000","last_unstale":"2024-05-08T06:36:43.108871+0000","last_undegraded":"2024-05-08T06:36:43.108871+0000","last_fullsized":"2024-05-08T06:36:43.108871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108766+0000","last_change":"2024-05-08T06:36:41.642429+0000","last_active":"2024-05-08T06:36:43.108766+0000","last_peered":"2024-05-08T06:36:43.108766+0000","last_clean":"2024-05-08T06:36:43.108766+0000","last_became_active":"2024-05-08T06:36:41.641956+0000","last_became_peered":"2024-05-08T06:36:41.641956+0000","last_unstale":"2024-05-08T06:36:43.108766+0000","last_undegraded":"2024-05-08T06:36:43.108766+0000","last_fullsized":"2024-05-08T06:36:43.108766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649200+0000","last_change":"2024-05-08T06:36:41.656423+0000","last_active":"2024-05-08T06:36:42.649200+0000","last_peered":"2024-05-08T06:36:42.649200+0000","last_clean":"2024-05-08T06:36:42.649200+0000","last_became_active":"2024-05-08T06:36:41.656154+0000","last_became_peered":"2024-05-08T06:36:41.656154+0000","last_unstale":"2024-05-08T06:36:42.649200+0000","last_undegraded":"2024-05-08T06:36:42.649200+0000","last_fullsized":"2024-05-08T06:36:42.649200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109681+0000","last_change":"2024-05-08T06:36:41.653378+0000","last_active":"2024-05-08T06:36:43.109681+0000","last_peered":"2024-05-08T06:36:43.109681+0000","last_clean":"2024-05-08T06:36:43.109681+0000","last_became_active":"2024-05-08T06:36:41.653027+0000","last_became_peered":"2024-05-08T06:36:41.653027+0000","last_unstale":"2024-05-08T06:36:43.109681+0000","last_undegraded":"2024-05-08T06:36:43.109681+0000","last_fullsized":"2024-05-08T06:36:43.109681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:02.654155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109919+0000","last_change":"2024-05-08T06:36:41.659055+0000","last_active":"2024-05-08T06:36:43.109919+0000","last_peered":"2024-05-08T06:36:43.109919+0000","last_clean":"2024-05-08T06:36:43.109919+0000","last_became_active":"2024-05-08T06:36:41.658774+0000","last_became_peered":"2024-05-08T06:36:41.658774+0000","last_unstale":"2024-05-08T06:36:43.109919+0000","last_undegraded":"2024-05-08T06:36:43.109919+0000","last_fullsized":"2024-05-08T06:36:43.109919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:38:12.052561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110931+0000","last_change":"2024-05-08T06:36:41.649669+0000","last_active":"2024-05-08T06:36:43.110931+0000","last_peered":"2024-05-08T06:36:43.110931+0000","last_clean":"2024-05-08T06:36:43.110931+0000","last_became_active":"2024-05-08T06:36:41.649391+0000","last_became_peered":"2024-05-08T06:36:41.649391+0000","last_unstale":"2024-05-08T06:36:43.110931+0000","last_undegraded":"2024-05-08T06:36:43.110931+0000","last_fullsized":"2024-05-08T06:36:43.110931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:04.579668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109443+0000","last_change":"2024-05-08T06:36:41.660852+0000","last_active":"2024-05-08T06:36:43.109443+0000","last_peered":"2024-05-08T06:36:43.109443+0000","last_clean":"2024-05-08T06:36:43.109443+0000","last_became_active":"2024-05-08T06:36:41.660573+0000","last_became_peered":"2024-05-08T06:36:41.660573+0000","last_unstale":"2024-05-08T06:36:43.109443+0000","last_undegraded":"2024-05-08T06:36:43.109443+0000","last_fullsized":"2024-05-08T06:36:43.109443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:02.654155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653881+0000","last_change":"2024-05-08T06:36:41.663211+0000","last_active":"2024-05-08T06:36:42.653881+0000","last_peered":"2024-05-08T06:36:42.653881+0000","last_clean":"2024-05-08T06:36:42.653881+0000","last_became_active":"2024-05-08T06:36:41.662936+0000","last_became_peered":"2024-05-08T06:36:41.662936+0000","last_unstale":"2024-05-08T06:36:42.653881+0000","last_undegraded":"2024-05-08T06:36:42.653881+0000","last_fullsized":"2024-05-08T06:36:42.653881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108430+0000","last_change":"2024-05-08T06:36:41.661453+0000","last_active":"2024-05-08T06:36:43.108430+0000","last_peered":"2024-05-08T06:36:43.108430+0000","last_clean":"2024-05-08T06:36:43.108430+0000","last_became_active":"2024-05-08T06:36:41.661146+0000","last_became_peered":"2024-05-08T06:36:41.661146+0000","last_unstale":"2024-05-08T06:36:43.108430+0000","last_undegraded":"2024-05-08T06:36:43.108430+0000","last_fullsized":"2024-05-08T06:36:43.108430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:11:54.263628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109377+0000","last_change":"2024-05-08T06:36:41.658664+0000","last_active":"2024-05-08T06:36:43.109377+0000","last_peered":"2024-05-08T06:36:43.109377+0000","last_clean":"2024-05-08T06:36:43.109377+0000","last_became_active":"2024-05-08T06:36:41.658352+0000","last_became_peered":"2024-05-08T06:36:41.658352+0000","last_unstale":"2024-05-08T06:36:43.109377+0000","last_undegraded":"2024-05-08T06:36:43.109377+0000","last_fullsized":"2024-05-08T06:36:43.109377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108694+0000","last_change":"2024-05-08T06:36:41.658967+0000","last_active":"2024-05-08T06:36:43.108694+0000","last_peered":"2024-05-08T06:36:43.108694+0000","last_clean":"2024-05-08T06:36:43.108694+0000","last_became_active":"2024-05-08T06:36:41.658387+0000","last_became_peered":"2024-05-08T06:36:41.658387+0000","last_unstale":"2024-05-08T06:36:43.108694+0000","last_undegraded":"2024-05-08T06:36:43.108694+0000","last_fullsized":"2024-05-08T06:36:43.108694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108892+0000","last_change":"2024-05-08T06:36:41.657102+0000","last_active":"2024-05-08T06:36:43.108892+0000","last_peered":"2024-05-08T06:36:43.108892+0000","last_clean":"2024-05-08T06:36:43.108892+0000","last_became_active":"2024-05-08T06:36:41.656844+0000","last_became_peered":"2024-05-08T06:36:41.656844+0000","last_unstale":"2024-05-08T06:36:43.108892+0000","last_undegraded":"2024-05-08T06:36:43.108892+0000","last_fullsized":"2024-05-08T06:36:43.108892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109150+0000","last_change":"2024-05-08T06:36:41.659592+0000","last_active":"2024-05-08T06:36:43.109150+0000","last_peered":"2024-05-08T06:36:43.109150+0000","last_clean":"2024-05-08T06:36:43.109150+0000","last_became_active":"2024-05-08T06:36:41.659295+0000","last_became_peered":"2024-05-08T06:36:41.659295+0000","last_unstale":"2024-05-08T06:36:43.109150+0000","last_undegraded":"2024-05-08T06:36:43.109150+0000","last_fullsized":"2024-05-08T06:36:43.109150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654479+0000","last_change":"2024-05-08T06:36:41.649572+0000","last_active":"2024-05-08T06:36:42.654479+0000","last_peered":"2024-05-08T06:36:42.654479+0000","last_clean":"2024-05-08T06:36:42.654479+0000","last_became_active":"2024-05-08T06:36:41.649302+0000","last_became_peered":"2024-05-08T06:36:41.649302+0000","last_unstale":"2024-05-08T06:36:42.654479+0000","last_undegraded":"2024-05-08T06:36:42.654479+0000","last_fullsized":"2024-05-08T06:36:42.654479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649400+0000","last_change":"2024-05-08T06:36:41.657182+0000","last_active":"2024-05-08T06:36:42.649400+0000","last_peered":"2024-05-08T06:36:42.649400+0000","last_clean":"2024-05-08T06:36:42.649400+0000","last_became_active":"2024-05-08T06:36:41.656919+0000","last_became_peered":"2024-05-08T06:36:41.656919+0000","last_unstale":"2024-05-08T06:36:42.649400+0000","last_undegraded":"2024-05-08T06:36:42.649400+0000","last_fullsized":"2024-05-08T06:36:42.649400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109121+0000","last_change":"2024-05-08T06:36:41.659147+0000","last_active":"2024-05-08T06:36:43.109121+0000","last_peered":"2024-05-08T06:36:43.109121+0000","last_clean":"2024-05-08T06:36:43.109121+0000","last_became_active":"2024-05-08T06:36:41.658880+0000","last_became_peered":"2024-05-08T06:36:41.658880+0000","last_unstale":"2024-05-08T06:36:43.109121+0000","last_undegraded":"2024-05-08T06:36:43.109121+0000","last_fullsized":"2024-05-08T06:36:43.109121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108909+0000","last_change":"2024-05-08T06:36:41.658925+0000","last_active":"2024-05-08T06:36:43.108909+0000","last_peered":"2024-05-08T06:36:43.108909+0000","last_clean":"2024-05-08T06:36:43.108909+0000","last_became_active":"2024-05-08T06:36:41.656879+0000","last_became_peered":"2024-05-08T06:36:41.656879+0000","last_unstale":"2024-05-08T06:36:43.108909+0000","last_undegraded":"2024-05-08T06:36:43.108909+0000","last_fullsized":"2024-05-08T06:36:43.108909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109551+0000","last_change":"2024-05-08T06:36:41.655125+0000","last_active":"2024-05-08T06:36:43.109551+0000","last_peered":"2024-05-08T06:36:43.109551+0000","last_clean":"2024-05-08T06:36:43.109551+0000","last_became_active":"2024-05-08T06:36:41.654817+0000","last_became_peered":"2024-05-08T06:36:41.654817+0000","last_unstale":"2024-05-08T06:36:43.109551+0000","last_undegraded":"2024-05-08T06:36:43.109551+0000","last_fullsized":"2024-05-08T06:36:43.109551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:42.444444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110786+0000","last_change":"2024-05-08T06:36:41.655361+0000","last_active":"2024-05-08T06:36:43.110786+0000","last_peered":"2024-05-08T06:36:43.110786+0000","last_clean":"2024-05-08T06:36:43.110786+0000","last_became_active":"2024-05-08T06:36:41.655085+0000","last_became_peered":"2024-05-08T06:36:41.655085+0000","last_unstale":"2024-05-08T06:36:43.110786+0000","last_undegraded":"2024-05-08T06:36:43.110786+0000","last_fullsized":"2024-05-08T06:36:43.110786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:26.842934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653569+0000","last_change":"2024-05-08T06:36:41.643791+0000","last_active":"2024-05-08T06:36:42.653569+0000","last_peered":"2024-05-08T06:36:42.653569+0000","last_clean":"2024-05-08T06:36:42.653569+0000","last_became_active":"2024-05-08T06:36:41.643506+0000","last_became_peered":"2024-05-08T06:36:41.643506+0000","last_unstale":"2024-05-08T06:36:42.653569+0000","last_undegraded":"2024-05-08T06:36:42.653569+0000","last_fullsized":"2024-05-08T06:36:42.653569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654375+0000","last_change":"2024-05-08T06:36:41.654106+0000","last_active":"2024-05-08T06:36:42.654375+0000","last_peered":"2024-05-08T06:36:42.654375+0000","last_clean":"2024-05-08T06:36:42.654375+0000","last_became_active":"2024-05-08T06:36:41.653822+0000","last_became_peered":"2024-05-08T06:36:41.653822+0000","last_unstale":"2024-05-08T06:36:42.654375+0000","last_undegraded":"2024-05-08T06:36:42.654375+0000","last_fullsized":"2024-05-08T06:36:42.654375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:51.358786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109209+0000","last_change":"2024-05-08T06:36:40.641349+0000","last_active":"2024-05-08T06:36:43.109209+0000","last_peered":"2024-05-08T06:36:43.109209+0000","last_clean":"2024-05-08T06:36:43.109209+0000","last_became_active":"2024-05-08T06:36:40.641086+0000","last_became_peered":"2024-05-08T06:36:40.641086+0000","last_unstale":"2024-05-08T06:36:43.109209+0000","last_undegraded":"2024-05-08T06:36:43.109209+0000","last_fullsized":"2024-05-08T06:36:43.109209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:39:09.616149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108579+0000","last_change":"2024-05-08T06:36:41.656120+0000","last_active":"2024-05-08T06:36:43.108579+0000","last_peered":"2024-05-08T06:36:43.108579+0000","last_clean":"2024-05-08T06:36:43.108579+0000","last_became_active":"2024-05-08T06:36:41.655843+0000","last_became_peered":"2024-05-08T06:36:41.655843+0000","last_unstale":"2024-05-08T06:36:43.108579+0000","last_undegraded":"2024-05-08T06:36:43.108579+0000","last_fullsized":"2024-05-08T06:36:43.108579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654066+0000","last_change":"2024-05-08T06:36:40.639740+0000","last_active":"2024-05-08T06:36:42.654066+0000","last_peered":"2024-05-08T06:36:42.654066+0000","last_clean":"2024-05-08T06:36:42.654066+0000","last_became_active":"2024-05-08T06:36:40.639370+0000","last_became_peered":"2024-05-08T06:36:40.639370+0000","last_unstale":"2024-05-08T06:36:42.654066+0000","last_undegraded":"2024-05-08T06:36:42.654066+0000","last_fullsized":"2024-05-08T06:36:42.654066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108684+0000","last_change":"2024-05-08T06:36:40.641983+0000","last_active":"2024-05-08T06:36:43.108684+0000","last_peered":"2024-05-08T06:36:43.108684+0000","last_clean":"2024-05-08T06:36:43.108684+0000","last_became_active":"2024-05-08T06:36:40.641666+0000","last_became_peered":"2024-05-08T06:36:40.641666+0000","last_unstale":"2024-05-08T06:36:43.108684+0000","last_undegraded":"2024-05-08T06:36:43.108684+0000","last_fullsized":"2024-05-08T06:36:43.108684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:47.997246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108516+0000","last_change":"2024-05-08T06:36:41.653257+0000","last_active":"2024-05-08T06:36:43.108516+0000","last_peered":"2024-05-08T06:36:43.108516+0000","last_clean":"2024-05-08T06:36:43.108516+0000","last_became_active":"2024-05-08T06:36:41.652900+0000","last_became_peered":"2024-05-08T06:36:41.652900+0000","last_unstale":"2024-05-08T06:36:43.108516+0000","last_undegraded":"2024-05-08T06:36:43.108516+0000","last_fullsized":"2024-05-08T06:36:43.108516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109272+0000","last_change":"2024-05-08T06:36:40.646856+0000","last_active":"2024-05-08T06:36:43.109272+0000","last_peered":"2024-05-08T06:36:43.109272+0000","last_clean":"2024-05-08T06:36:43.109272+0000","last_became_active":"2024-05-08T06:36:40.644640+0000","last_became_peered":"2024-05-08T06:36:40.644640+0000","last_unstale":"2024-05-08T06:36:43.109272+0000","last_undegraded":"2024-05-08T06:36:43.109272+0000","last_fullsized":"2024-05-08T06:36:43.109272+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649183+0000","last_change":"2024-05-08T06:36:41.656744+0000","last_active":"2024-05-08T06:36:42.649183+0000","last_peered":"2024-05-08T06:36:42.649183+0000","last_clean":"2024-05-08T06:36:42.649183+0000","last_became_active":"2024-05-08T06:36:41.656483+0000","last_became_peered":"2024-05-08T06:36:41.656483+0000","last_unstale":"2024-05-08T06:36:42.649183+0000","last_undegraded":"2024-05-08T06:36:42.649183+0000","last_fullsized":"2024-05-08T06:36:42.649183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653768+0000","last_change":"2024-05-08T06:36:40.648551+0000","last_active":"2024-05-08T06:36:42.653768+0000","last_peered":"2024-05-08T06:36:42.653768+0000","last_clean":"2024-05-08T06:36:42.653768+0000","last_became_active":"2024-05-08T06:36:40.648265+0000","last_became_peered":"2024-05-08T06:36:40.648265+0000","last_unstale":"2024-05-08T06:36:42.653768+0000","last_undegraded":"2024-05-08T06:36:42.653768+0000","last_fullsized":"2024-05-08T06:36:42.653768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109024+0000","last_change":"2024-05-08T06:36:41.658637+0000","last_active":"2024-05-08T06:36:43.109024+0000","last_peered":"2024-05-08T06:36:43.109024+0000","last_clean":"2024-05-08T06:36:43.109024+0000","last_became_active":"2024-05-08T06:36:41.658333+0000","last_became_peered":"2024-05-08T06:36:41.658333+0000","last_unstale":"2024-05-08T06:36:43.109024+0000","last_undegraded":"2024-05-08T06:36:43.109024+0000","last_fullsized":"2024-05-08T06:36:43.109024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109045+0000","last_change":"2024-05-08T06:36:40.647645+0000","last_active":"2024-05-08T06:36:43.109045+0000","last_peered":"2024-05-08T06:36:43.109045+0000","last_clean":"2024-05-08T06:36:43.109045+0000","last_became_active":"2024-05-08T06:36:40.647368+0000","last_became_peered":"2024-05-08T06:36:40.647368+0000","last_unstale":"2024-05-08T06:36:43.109045+0000","last_undegraded":"2024-05-08T06:36:43.109045+0000","last_fullsized":"2024-05-08T06:36:43.109045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109122+0000","last_change":"2024-05-08T06:36:41.662012+0000","last_active":"2024-05-08T06:36:43.109122+0000","last_peered":"2024-05-08T06:36:43.109122+0000","last_clean":"2024-05-08T06:36:43.109122+0000","last_became_active":"2024-05-08T06:36:41.661780+0000","last_became_peered":"2024-05-08T06:36:41.661780+0000","last_unstale":"2024-05-08T06:36:43.109122+0000","last_undegraded":"2024-05-08T06:36:43.109122+0000","last_fullsized":"2024-05-08T06:36:43.109122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649372+0000","last_change":"2024-05-08T06:36:40.649405+0000","last_active":"2024-05-08T06:36:42.649372+0000","last_peered":"2024-05-08T06:36:42.649372+0000","last_clean":"2024-05-08T06:36:42.649372+0000","last_became_active":"2024-05-08T06:36:40.649172+0000","last_became_peered":"2024-05-08T06:36:40.649172+0000","last_unstale":"2024-05-08T06:36:42.649372+0000","last_undegraded":"2024-05-08T06:36:42.649372+0000","last_fullsized":"2024-05-08T06:36:42.649372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654679+0000","last_change":"2024-05-08T06:36:41.654141+0000","last_active":"2024-05-08T06:36:42.654679+0000","last_peered":"2024-05-08T06:36:42.654679+0000","last_clean":"2024-05-08T06:36:42.654679+0000","last_became_active":"2024-05-08T06:36:41.653723+0000","last_became_peered":"2024-05-08T06:36:41.653723+0000","last_unstale":"2024-05-08T06:36:42.654679+0000","last_undegraded":"2024-05-08T06:36:42.654679+0000","last_fullsized":"2024-05-08T06:36:42.654679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:04.579668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654000+0000","last_change":"2024-05-08T06:36:40.641783+0000","last_active":"2024-05-08T06:36:42.654000+0000","last_peered":"2024-05-08T06:36:42.654000+0000","last_clean":"2024-05-08T06:36:42.654000+0000","last_became_active":"2024-05-08T06:36:40.641270+0000","last_became_peered":"2024-05-08T06:36:40.641270+0000","last_unstale":"2024-05-08T06:36:42.654000+0000","last_undegraded":"2024-05-08T06:36:42.654000+0000","last_fullsized":"2024-05-08T06:36:42.654000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108979+0000","last_change":"2024-05-08T06:36:41.657579+0000","last_active":"2024-05-08T06:36:43.108979+0000","last_peered":"2024-05-08T06:36:43.108979+0000","last_clean":"2024-05-08T06:36:43.108979+0000","last_became_active":"2024-05-08T06:36:41.657299+0000","last_became_peered":"2024-05-08T06:36:41.657299+0000","last_unstale":"2024-05-08T06:36:43.108979+0000","last_undegraded":"2024-05-08T06:36:43.108979+0000","last_fullsized":"2024-05-08T06:36:43.108979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:02.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108685+0000","last_change":"2024-05-08T06:36:40.641651+0000","last_active":"2024-05-08T06:36:43.108685+0000","last_peered":"2024-05-08T06:36:43.108685+0000","last_clean":"2024-05-08T06:36:43.108685+0000","last_became_active":"2024-05-08T06:36:40.641168+0000","last_became_peered":"2024-05-08T06:36:40.641168+0000","last_unstale":"2024-05-08T06:36:43.108685+0000","last_undegraded":"2024-05-08T06:36:43.108685+0000","last_fullsized":"2024-05-08T06:36:43.108685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:47.997246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108903+0000","last_change":"2024-05-08T06:36:41.661356+0000","last_active":"2024-05-08T06:36:43.108903+0000","last_peered":"2024-05-08T06:36:43.108903+0000","last_clean":"2024-05-08T06:36:43.108903+0000","last_became_active":"2024-05-08T06:36:41.661085+0000","last_became_peered":"2024-05-08T06:36:41.661085+0000","last_unstale":"2024-05-08T06:36:43.108903+0000","last_undegraded":"2024-05-08T06:36:43.108903+0000","last_fullsized":"2024-05-08T06:36:43.108903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109770+0000","last_change":"2024-05-08T06:36:40.646694+0000","last_active":"2024-05-08T06:36:43.109770+0000","last_peered":"2024-05-08T06:36:43.109770+0000","last_clean":"2024-05-08T06:36:43.109770+0000","last_became_active":"2024-05-08T06:36:40.645833+0000","last_became_peered":"2024-05-08T06:36:40.645833+0000","last_unstale":"2024-05-08T06:36:43.109770+0000","last_undegraded":"2024-05-08T06:36:43.109770+0000","last_fullsized":"2024-05-08T06:36:43.109770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109418+0000","last_change":"2024-05-08T06:36:41.668787+0000","last_active":"2024-05-08T06:36:43.109418+0000","last_peered":"2024-05-08T06:36:43.109418+0000","last_clean":"2024-05-08T06:36:43.109418+0000","last_became_active":"2024-05-08T06:36:41.668489+0000","last_became_peered":"2024-05-08T06:36:41.668489+0000","last_unstale":"2024-05-08T06:36:43.109418+0000","last_undegraded":"2024-05-08T06:36:43.109418+0000","last_fullsized":"2024-05-08T06:36:43.109418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.395237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653600+0000","last_change":"2024-05-08T06:36:40.640082+0000","last_active":"2024-05-08T06:36:42.653600+0000","last_peered":"2024-05-08T06:36:42.653600+0000","last_clean":"2024-05-08T06:36:42.653600+0000","last_became_active":"2024-05-08T06:36:40.638558+0000","last_became_peered":"2024-05-08T06:36:40.638558+0000","last_unstale":"2024-05-08T06:36:42.653600+0000","last_undegraded":"2024-05-08T06:36:42.653600+0000","last_fullsized":"2024-05-08T06:36:42.653600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653523+0000","last_change":"2024-05-08T06:36:41.643855+0000","last_active":"2024-05-08T06:36:42.653523+0000","last_peered":"2024-05-08T06:36:42.653523+0000","last_clean":"2024-05-08T06:36:42.653523+0000","last_became_active":"2024-05-08T06:36:41.643117+0000","last_became_peered":"2024-05-08T06:36:41.643117+0000","last_unstale":"2024-05-08T06:36:42.653523+0000","last_undegraded":"2024-05-08T06:36:42.653523+0000","last_fullsized":"2024-05-08T06:36:42.653523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110713+0000","last_change":"2024-05-08T06:36:40.642303+0000","last_active":"2024-05-08T06:36:43.110713+0000","last_peered":"2024-05-08T06:36:43.110713+0000","last_clean":"2024-05-08T06:36:43.110713+0000","last_became_active":"2024-05-08T06:36:40.642042+0000","last_became_peered":"2024-05-08T06:36:40.642042+0000","last_unstale":"2024-05-08T06:36:43.110713+0000","last_undegraded":"2024-05-08T06:36:43.110713+0000","last_fullsized":"2024-05-08T06:36:43.110713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109058+0000","last_change":"2024-05-08T06:36:41.660559+0000","last_active":"2024-05-08T06:36:43.109058+0000","last_peered":"2024-05-08T06:36:43.109058+0000","last_clean":"2024-05-08T06:36:43.109058+0000","last_became_active":"2024-05-08T06:36:41.660282+0000","last_became_peered":"2024-05-08T06:36:41.660282+0000","last_unstale":"2024-05-08T06:36:43.109058+0000","last_undegraded":"2024-05-08T06:36:43.109058+0000","last_fullsized":"2024-05-08T06:36:43.109058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:49.740862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654374+0000","last_change":"2024-05-08T06:36:40.646655+0000","last_active":"2024-05-08T06:36:42.654374+0000","last_peered":"2024-05-08T06:36:42.654374+0000","last_clean":"2024-05-08T06:36:42.654374+0000","last_became_active":"2024-05-08T06:36:40.646346+0000","last_became_peered":"2024-05-08T06:36:40.646346+0000","last_unstale":"2024-05-08T06:36:42.654374+0000","last_undegraded":"2024-05-08T06:36:42.654374+0000","last_fullsized":"2024-05-08T06:36:42.654374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109890+0000","last_change":"2024-05-08T06:36:41.659817+0000","last_active":"2024-05-08T06:36:43.109890+0000","last_peered":"2024-05-08T06:36:43.109890+0000","last_clean":"2024-05-08T06:36:43.109890+0000","last_became_active":"2024-05-08T06:36:41.659524+0000","last_became_peered":"2024-05-08T06:36:41.659524+0000","last_unstale":"2024-05-08T06:36:43.109890+0000","last_undegraded":"2024-05-08T06:36:43.109890+0000","last_fullsized":"2024-05-08T06:36:43.109890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:38:12.052561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110881+0000","last_change":"2024-05-08T06:36:40.641491+0000","last_active":"2024-05-08T06:36:43.110881+0000","last_peered":"2024-05-08T06:36:43.110881+0000","last_clean":"2024-05-08T06:36:43.110881+0000","last_became_active":"2024-05-08T06:36:40.641207+0000","last_became_peered":"2024-05-08T06:36:40.641207+0000","last_unstale":"2024-05-08T06:36:43.110881+0000","last_undegraded":"2024-05-08T06:36:43.110881+0000","last_fullsized":"2024-05-08T06:36:43.110881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110049+0000","last_change":"2024-05-08T06:36:41.667992+0000","last_active":"2024-05-08T06:36:43.110049+0000","last_peered":"2024-05-08T06:36:43.110049+0000","last_clean":"2024-05-08T06:36:43.110049+0000","last_became_active":"2024-05-08T06:36:41.667733+0000","last_became_peered":"2024-05-08T06:36:41.667733+0000","last_unstale":"2024-05-08T06:36:43.110049+0000","last_undegraded":"2024-05-08T06:36:43.110049+0000","last_fullsized":"2024-05-08T06:36:43.110049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:41:00.520597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649448+0000","last_change":"2024-05-08T06:36:40.646069+0000","last_active":"2024-05-08T06:36:42.649448+0000","last_peered":"2024-05-08T06:36:42.649448+0000","last_clean":"2024-05-08T06:36:42.649448+0000","last_became_active":"2024-05-08T06:36:40.645769+0000","last_became_peered":"2024-05-08T06:36:40.645769+0000","last_unstale":"2024-05-08T06:36:42.649448+0000","last_undegraded":"2024-05-08T06:36:42.649448+0000","last_fullsized":"2024-05-08T06:36:42.649448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:25.837342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109628+0000","last_change":"2024-05-08T06:36:41.660346+0000","last_active":"2024-05-08T06:36:43.109628+0000","last_peered":"2024-05-08T06:36:43.109628+0000","last_clean":"2024-05-08T06:36:43.109628+0000","last_became_active":"2024-05-08T06:36:41.660028+0000","last_became_peered":"2024-05-08T06:36:41.660028+0000","last_unstale":"2024-05-08T06:36:43.109628+0000","last_undegraded":"2024-05-08T06:36:43.109628+0000","last_fullsized":"2024-05-08T06:36:43.109628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:14.886723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110847+0000","last_change":"2024-05-08T06:36:40.647112+0000","last_active":"2024-05-08T06:36:43.110847+0000","last_peered":"2024-05-08T06:36:43.110847+0000","last_clean":"2024-05-08T06:36:43.110847+0000","last_became_active":"2024-05-08T06:36:40.646804+0000","last_became_peered":"2024-05-08T06:36:40.646804+0000","last_unstale":"2024-05-08T06:36:43.110847+0000","last_undegraded":"2024-05-08T06:36:43.110847+0000","last_fullsized":"2024-05-08T06:36:43.110847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109148+0000","last_change":"2024-05-08T06:36:41.663875+0000","last_active":"2024-05-08T06:36:43.109148+0000","last_peered":"2024-05-08T06:36:43.109148+0000","last_clean":"2024-05-08T06:36:43.109148+0000","last_became_active":"2024-05-08T06:36:41.663595+0000","last_became_peered":"2024-05-08T06:36:41.663595+0000","last_unstale":"2024-05-08T06:36:43.109148+0000","last_undegraded":"2024-05-08T06:36:43.109148+0000","last_fullsized":"2024-05-08T06:36:43.109148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:42.444444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108725+0000","last_change":"2024-05-08T06:36:31.710287+0000","last_active":"2024-05-08T06:36:43.108725+0000","last_peered":"2024-05-08T06:36:43.108725+0000","last_clean":"2024-05-08T06:36:43.108725+0000","last_became_active":"2024-05-08T06:36:31.709270+0000","last_became_peered":"2024-05-08T06:36:31.709270+0000","last_unstale":"2024-05-08T06:36:43.108725+0000","last_undegraded":"2024-05-08T06:36:43.108725+0000","last_fullsized":"2024-05-08T06:36:43.108725+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_clean_scrub_stamp":"2024-05-08T06:36:30.122918+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:42.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108783+0000","last_change":"2024-05-08T06:36:40.645536+0000","last_active":"2024-05-08T06:36:43.108783+0000","last_peered":"2024-05-08T06:36:43.108783+0000","last_clean":"2024-05-08T06:36:43.108783+0000","last_became_active":"2024-05-08T06:36:40.644256+0000","last_became_peered":"2024-05-08T06:36:40.644256+0000","last_unstale":"2024-05-08T06:36:43.108783+0000","last_undegraded":"2024-05-08T06:36:43.108783+0000","last_fullsized":"2024-05-08T06:36:43.108783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109026+0000","last_change":"2024-05-08T06:36:41.655301+0000","last_active":"2024-05-08T06:36:43.109026+0000","last_peered":"2024-05-08T06:36:43.109026+0000","last_clean":"2024-05-08T06:36:43.109026+0000","last_became_active":"2024-05-08T06:36:41.655090+0000","last_became_peered":"2024-05-08T06:36:41.655090+0000","last_unstale":"2024-05-08T06:36:43.109026+0000","last_undegraded":"2024-05-08T06:36:43.109026+0000","last_fullsized":"2024-05-08T06:36:43.109026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108653+0000","last_change":"2024-05-08T06:36:40.647913+0000","last_active":"2024-05-08T06:36:43.108653+0000","last_peered":"2024-05-08T06:36:43.108653+0000","last_clean":"2024-05-08T06:36:43.108653+0000","last_became_active":"2024-05-08T06:36:40.647638+0000","last_became_peered":"2024-05-08T06:36:40.647638+0000","last_unstale":"2024-05-08T06:36:43.108653+0000","last_undegraded":"2024-05-08T06:36:43.108653+0000","last_fullsized":"2024-05-08T06:36:43.108653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:29:44.646218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108285+0000","last_change":"2024-05-08T06:36:41.658563+0000","last_active":"2024-05-08T06:36:43.108285+0000","last_peered":"2024-05-08T06:36:43.108285+0000","last_clean":"2024-05-08T06:36:43.108285+0000","last_became_active":"2024-05-08T06:36:41.657157+0000","last_became_peered":"2024-05-08T06:36:41.657157+0000","last_unstale":"2024-05-08T06:36:43.108285+0000","last_undegraded":"2024-05-08T06:36:43.108285+0000","last_fullsized":"2024-05-08T06:36:43.108285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:30:14.798179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110812+0000","last_change":"2024-05-08T06:36:40.642187+0000","last_active":"2024-05-08T06:36:43.110812+0000","last_peered":"2024-05-08T06:36:43.110812+0000","last_clean":"2024-05-08T06:36:43.110812+0000","last_became_active":"2024-05-08T06:36:40.641906+0000","last_became_peered":"2024-05-08T06:36:40.641906+0000","last_unstale":"2024-05-08T06:36:43.110812+0000","last_undegraded":"2024-05-08T06:36:43.110812+0000","last_fullsized":"2024-05-08T06:36:43.110812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649176+0000","last_change":"2024-05-08T06:36:41.642328+0000","last_active":"2024-05-08T06:36:42.649176+0000","last_peered":"2024-05-08T06:36:42.649176+0000","last_clean":"2024-05-08T06:36:42.649176+0000","last_became_active":"2024-05-08T06:36:41.642053+0000","last_became_peered":"2024-05-08T06:36:41.642053+0000","last_unstale":"2024-05-08T06:36:42.649176+0000","last_undegraded":"2024-05-08T06:36:42.649176+0000","last_fullsized":"2024-05-08T06:36:42.649176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109887+0000","last_change":"2024-05-08T06:36:40.646469+0000","last_active":"2024-05-08T06:36:43.109887+0000","last_peered":"2024-05-08T06:36:43.109887+0000","last_clean":"2024-05-08T06:36:43.109887+0000","last_became_active":"2024-05-08T06:36:40.646207+0000","last_became_peered":"2024-05-08T06:36:40.646207+0000","last_unstale":"2024-05-08T06:36:43.109887+0000","last_undegraded":"2024-05-08T06:36:43.109887+0000","last_fullsized":"2024-05-08T06:36:43.109887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654706+0000","last_change":"2024-05-08T06:36:41.654729+0000","last_active":"2024-05-08T06:36:42.654706+0000","last_peered":"2024-05-08T06:36:42.654706+0000","last_clean":"2024-05-08T06:36:42.654706+0000","last_became_active":"2024-05-08T06:36:41.654466+0000","last_became_peered":"2024-05-08T06:36:41.654466+0000","last_unstale":"2024-05-08T06:36:42.654706+0000","last_undegraded":"2024-05-08T06:36:42.654706+0000","last_fullsized":"2024-05-08T06:36:42.654706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:44.798191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649223+0000","last_change":"2024-05-08T06:36:40.647304+0000","last_active":"2024-05-08T06:36:42.649223+0000","last_peered":"2024-05-08T06:36:42.649223+0000","last_clean":"2024-05-08T06:36:42.649223+0000","last_became_active":"2024-05-08T06:36:40.647010+0000","last_became_peered":"2024-05-08T06:36:40.647010+0000","last_unstale":"2024-05-08T06:36:42.649223+0000","last_undegraded":"2024-05-08T06:36:42.649223+0000","last_fullsized":"2024-05-08T06:36:42.649223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:50.353194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109655+0000","last_change":"2024-05-08T06:36:41.656064+0000","last_active":"2024-05-08T06:36:43.109655+0000","last_peered":"2024-05-08T06:36:43.109655+0000","last_clean":"2024-05-08T06:36:43.109655+0000","last_became_active":"2024-05-08T06:36:41.655714+0000","last_became_peered":"2024-05-08T06:36:41.655714+0000","last_unstale":"2024-05-08T06:36:43.109655+0000","last_undegraded":"2024-05-08T06:36:43.109655+0000","last_fullsized":"2024-05-08T06:36:43.109655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.395237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109737+0000","last_change":"2024-05-08T06:36:40.648662+0000","last_active":"2024-05-08T06:36:43.109737+0000","last_peered":"2024-05-08T06:36:43.109737+0000","last_clean":"2024-05-08T06:36:43.109737+0000","last_became_active":"2024-05-08T06:36:40.648367+0000","last_became_peered":"2024-05-08T06:36:40.648367+0000","last_unstale":"2024-05-08T06:36:43.109737+0000","last_undegraded":"2024-05-08T06:36:43.109737+0000","last_fullsized":"2024-05-08T06:36:43.109737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:25.837342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649254+0000","last_change":"2024-05-08T06:36:41.642330+0000","last_active":"2024-05-08T06:36:42.649254+0000","last_peered":"2024-05-08T06:36:42.649254+0000","last_clean":"2024-05-08T06:36:42.649254+0000","last_became_active":"2024-05-08T06:36:41.642059+0000","last_became_peered":"2024-05-08T06:36:41.642059+0000","last_unstale":"2024-05-08T06:36:42.649254+0000","last_undegraded":"2024-05-08T06:36:42.649254+0000","last_fullsized":"2024-05-08T06:36:42.649254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649350+0000","last_change":"2024-05-08T06:36:40.646675+0000","last_active":"2024-05-08T06:36:42.649350+0000","last_peered":"2024-05-08T06:36:42.649350+0000","last_clean":"2024-05-08T06:36:42.649350+0000","last_became_active":"2024-05-08T06:36:40.646346+0000","last_became_peered":"2024-05-08T06:36:40.646346+0000","last_unstale":"2024-05-08T06:36:42.649350+0000","last_undegraded":"2024-05-08T06:36:42.649350+0000","last_fullsized":"2024-05-08T06:36:42.649350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:03.574076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109189+0000","last_change":"2024-05-08T06:36:41.658633+0000","last_active":"2024-05-08T06:36:43.109189+0000","last_peered":"2024-05-08T06:36:43.109189+0000","last_clean":"2024-05-08T06:36:43.109189+0000","last_became_active":"2024-05-08T06:36:41.658294+0000","last_became_peered":"2024-05-08T06:36:41.658294+0000","last_unstale":"2024-05-08T06:36:43.109189+0000","last_undegraded":"2024-05-08T06:36:43.109189+0000","last_fullsized":"2024-05-08T06:36:43.109189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653882+0000","last_change":"2024-05-08T06:36:40.639547+0000","last_active":"2024-05-08T06:36:42.653882+0000","last_peered":"2024-05-08T06:36:42.653882+0000","last_clean":"2024-05-08T06:36:42.653882+0000","last_became_active":"2024-05-08T06:36:40.639240+0000","last_became_peered":"2024-05-08T06:36:40.639240+0000","last_unstale":"2024-05-08T06:36:42.653882+0000","last_undegraded":"2024-05-08T06:36:42.653882+0000","last_fullsized":"2024-05-08T06:36:42.653882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108530+0000","last_change":"2024-05-08T06:36:41.648709+0000","last_active":"2024-05-08T06:36:43.108530+0000","last_peered":"2024-05-08T06:36:43.108530+0000","last_clean":"2024-05-08T06:36:43.108530+0000","last_became_active":"2024-05-08T06:36:41.648406+0000","last_became_peered":"2024-05-08T06:36:41.648406+0000","last_unstale":"2024-05-08T06:36:43.108530+0000","last_undegraded":"2024-05-08T06:36:43.108530+0000","last_fullsized":"2024-05-08T06:36:43.108530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108653+0000","last_change":"2024-05-08T06:36:40.648721+0000","last_active":"2024-05-08T06:36:43.108653+0000","last_peered":"2024-05-08T06:36:43.108653+0000","last_clean":"2024-05-08T06:36:43.108653+0000","last_became_active":"2024-05-08T06:36:40.648429+0000","last_became_peered":"2024-05-08T06:36:40.648429+0000","last_unstale":"2024-05-08T06:36:43.108653+0000","last_undegraded":"2024-05-08T06:36:43.108653+0000","last_fullsized":"2024-05-08T06:36:43.108653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:43.792599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654290+0000","last_change":"2024-05-08T06:36:41.655289+0000","last_active":"2024-05-08T06:36:42.654290+0000","last_peered":"2024-05-08T06:36:42.654290+0000","last_clean":"2024-05-08T06:36:42.654290+0000","last_became_active":"2024-05-08T06:36:41.655022+0000","last_became_peered":"2024-05-08T06:36:41.655022+0000","last_unstale":"2024-05-08T06:36:42.654290+0000","last_undegraded":"2024-05-08T06:36:42.654290+0000","last_fullsized":"2024-05-08T06:36:42.654290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:57.405934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649256+0000","last_change":"2024-05-08T06:36:40.645335+0000","last_active":"2024-05-08T06:36:42.649256+0000","last_peered":"2024-05-08T06:36:42.649256+0000","last_clean":"2024-05-08T06:36:42.649256+0000","last_became_active":"2024-05-08T06:36:40.645019+0000","last_became_peered":"2024-05-08T06:36:40.645019+0000","last_unstale":"2024-05-08T06:36:42.649256+0000","last_undegraded":"2024-05-08T06:36:42.649256+0000","last_fullsized":"2024-05-08T06:36:42.649256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108851+0000","last_change":"2024-05-08T06:36:41.653910+0000","last_active":"2024-05-08T06:36:43.108851+0000","last_peered":"2024-05-08T06:36:43.108851+0000","last_clean":"2024-05-08T06:36:43.108851+0000","last_became_active":"2024-05-08T06:36:41.653567+0000","last_became_peered":"2024-05-08T06:36:41.653567+0000","last_unstale":"2024-05-08T06:36:43.108851+0000","last_undegraded":"2024-05-08T06:36:43.108851+0000","last_fullsized":"2024-05-08T06:36:43.108851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649426+0000","last_change":"2024-05-08T06:36:40.642880+0000","last_active":"2024-05-08T06:36:42.649426+0000","last_peered":"2024-05-08T06:36:42.649426+0000","last_clean":"2024-05-08T06:36:42.649426+0000","last_became_active":"2024-05-08T06:36:40.642605+0000","last_became_peered":"2024-05-08T06:36:40.642605+0000","last_unstale":"2024-05-08T06:36:42.649426+0000","last_undegraded":"2024-05-08T06:36:42.649426+0000","last_fullsized":"2024-05-08T06:36:42.649426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109118+0000","last_change":"2024-05-08T06:36:41.654124+0000","last_active":"2024-05-08T06:36:43.109118+0000","last_peered":"2024-05-08T06:36:43.109118+0000","last_clean":"2024-05-08T06:36:43.109118+0000","last_became_active":"2024-05-08T06:36:41.653837+0000","last_became_peered":"2024-05-08T06:36:41.653837+0000","last_unstale":"2024-05-08T06:36:43.109118+0000","last_undegraded":"2024-05-08T06:36:43.109118+0000","last_fullsized":"2024-05-08T06:36:43.109118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109611+0000","last_change":"2024-05-08T06:36:40.644938+0000","last_active":"2024-05-08T06:36:43.109611+0000","last_peered":"2024-05-08T06:36:43.109611+0000","last_clean":"2024-05-08T06:36:43.109611+0000","last_became_active":"2024-05-08T06:36:40.644549+0000","last_became_peered":"2024-05-08T06:36:40.644549+0000","last_unstale":"2024-05-08T06:36:43.109611+0000","last_undegraded":"2024-05-08T06:36:43.109611+0000","last_fullsized":"2024-05-08T06:36:43.109611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:50.353194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653954+0000","last_change":"2024-05-08T06:36:41.659401+0000","last_active":"2024-05-08T06:36:42.653954+0000","last_peered":"2024-05-08T06:36:42.653954+0000","last_clean":"2024-05-08T06:36:42.653954+0000","last_became_active":"2024-05-08T06:36:41.659063+0000","last_became_peered":"2024-05-08T06:36:41.659063+0000","last_unstale":"2024-05-08T06:36:42.653954+0000","last_undegraded":"2024-05-08T06:36:42.653954+0000","last_fullsized":"2024-05-08T06:36:42.653954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109449+0000","last_change":"2024-05-08T06:36:41.655858+0000","last_active":"2024-05-08T06:36:43.109449+0000","last_peered":"2024-05-08T06:36:43.109449+0000","last_clean":"2024-05-08T06:36:43.109449+0000","last_became_active":"2024-05-08T06:36:41.655569+0000","last_became_peered":"2024-05-08T06:36:41.655569+0000","last_unstale":"2024-05-08T06:36:43.109449+0000","last_undegraded":"2024-05-08T06:36:43.109449+0000","last_fullsized":"2024-05-08T06:36:43.109449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:02.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109476+0000","last_change":"2024-05-08T06:36:40.646643+0000","last_active":"2024-05-08T06:36:43.109476+0000","last_peered":"2024-05-08T06:36:43.109476+0000","last_clean":"2024-05-08T06:36:43.109476+0000","last_became_active":"2024-05-08T06:36:40.646319+0000","last_became_peered":"2024-05-08T06:36:40.646319+0000","last_unstale":"2024-05-08T06:36:43.109476+0000","last_undegraded":"2024-05-08T06:36:43.109476+0000","last_fullsized":"2024-05-08T06:36:43.109476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:03.574076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109441+0000","last_change":"2024-05-08T06:36:41.656383+0000","last_active":"2024-05-08T06:36:43.109441+0000","last_peered":"2024-05-08T06:36:43.109441+0000","last_clean":"2024-05-08T06:36:43.109441+0000","last_became_active":"2024-05-08T06:36:41.656059+0000","last_became_peered":"2024-05-08T06:36:41.656059+0000","last_unstale":"2024-05-08T06:36:43.109441+0000","last_undegraded":"2024-05-08T06:36:43.109441+0000","last_fullsized":"2024-05-08T06:36:43.109441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:49.740862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109265+0000","last_change":"2024-05-08T06:36:40.646560+0000","last_active":"2024-05-08T06:36:43.109265+0000","last_peered":"2024-05-08T06:36:43.109265+0000","last_clean":"2024-05-08T06:36:43.109265+0000","last_became_active":"2024-05-08T06:36:40.646285+0000","last_became_peered":"2024-05-08T06:36:40.646285+0000","last_unstale":"2024-05-08T06:36:43.109265+0000","last_undegraded":"2024-05-08T06:36:43.109265+0000","last_fullsized":"2024-05-08T06:36:43.109265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108439+0000","last_change":"2024-05-08T06:36:41.661975+0000","last_active":"2024-05-08T06:36:43.108439+0000","last_peered":"2024-05-08T06:36:43.108439+0000","last_clean":"2024-05-08T06:36:43.108439+0000","last_became_active":"2024-05-08T06:36:41.661759+0000","last_became_peered":"2024-05-08T06:36:41.661759+0000","last_unstale":"2024-05-08T06:36:43.108439+0000","last_undegraded":"2024-05-08T06:36:43.108439+0000","last_fullsized":"2024-05-08T06:36:43.108439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109215+0000","last_change":"2024-05-08T06:36:40.645426+0000","last_active":"2024-05-08T06:36:43.109215+0000","last_peered":"2024-05-08T06:36:43.109215+0000","last_clean":"2024-05-08T06:36:43.109215+0000","last_became_active":"2024-05-08T06:36:40.645194+0000","last_became_peered":"2024-05-08T06:36:40.645194+0000","last_unstale":"2024-05-08T06:36:43.109215+0000","last_undegraded":"2024-05-08T06:36:43.109215+0000","last_fullsized":"2024-05-08T06:36:43.109215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109153+0000","last_change":"2024-05-08T06:36:41.660148+0000","last_active":"2024-05-08T06:36:43.109153+0000","last_peered":"2024-05-08T06:36:43.109153+0000","last_clean":"2024-05-08T06:36:43.109153+0000","last_became_active":"2024-05-08T06:36:41.659907+0000","last_became_peered":"2024-05-08T06:36:41.659907+0000","last_unstale":"2024-05-08T06:36:43.109153+0000","last_undegraded":"2024-05-08T06:36:43.109153+0000","last_fullsized":"2024-05-08T06:36:43.109153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:45:07.961465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109158+0000","last_change":"2024-05-08T06:36:40.645379+0000","last_active":"2024-05-08T06:36:43.109158+0000","last_peered":"2024-05-08T06:36:43.109158+0000","last_clean":"2024-05-08T06:36:43.109158+0000","last_became_active":"2024-05-08T06:36:40.645091+0000","last_became_peered":"2024-05-08T06:36:40.645091+0000","last_unstale":"2024-05-08T06:36:43.109158+0000","last_undegraded":"2024-05-08T06:36:43.109158+0000","last_fullsized":"2024-05-08T06:36:43.109158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:36:50.395 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T06:36:50.395 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T06:36:50.395 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-08T06:36:50.396 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-08T06:36:50.396 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-08T06:36:50.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.532+0000 7f7e9919d640 1 Processor -- start 2024-05-08T06:36:50.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.532+0000 7f7e9919d640 1 -- start start 2024-05-08T06:36:50.534 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.532+0000 7f7e9919d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e941423c0 0x7f7e94142a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.534 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.532+0000 7f7e9919d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7e94163800 0x7f7e940d32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.534 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.532+0000 7f7e9919d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7e940d3830 0x7f7e94144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.534 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.532+0000 7f7e9919d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f7e941608b0 con 0x7f7e940d3830 2024-05-08T06:36:50.534 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.532+0000 7f7e9919d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f7e940d3ca0 con 0x7f7e94163800 2024-05-08T06:36:50.534 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.532+0000 7f7e9919d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7e94142f50 con 0x7f7e941423c0 2024-05-08T06:36:50.534 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.533+0000 7f7e9359e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7e940d3830 0x7f7e94144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.534 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.533+0000 7f7e9359e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7e940d3830 0x7f7e94144dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44330/0 (socket says 172.21.15.163:44330) 2024-05-08T06:36:50.534 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.533+0000 7f7e9359e640 1 -- 172.21.15.163:0/3395199891 learned_addr learned my addr 172.21.15.163:0/3395199891 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:50.534 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.533+0000 7f7e92d9d640 1 --2- 172.21.15.163:0/3395199891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e941423c0 0x7f7e94142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.534 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.533+0000 7f7e9259c640 1 --2- 172.21.15.163:0/3395199891 >> v2:172.21.15.192:3300/0 conn(0x7f7e94163800 0x7f7e940d32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.533+0000 7f7e9359e640 1 -- 172.21.15.163:0/3395199891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e941423c0 msgr2=0x7f7e94142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:50.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.533+0000 7f7e9359e640 1 --2- 172.21.15.163:0/3395199891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e941423c0 0x7f7e94142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.533+0000 7f7e9359e640 1 -- 172.21.15.163:0/3395199891 >> v2:172.21.15.192:3300/0 conn(0x7f7e94163800 msgr2=0x7f7e940d32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:50.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.533+0000 7f7e9359e640 1 --2- 172.21.15.163:0/3395199891 >> v2:172.21.15.192:3300/0 conn(0x7f7e94163800 0x7f7e940d32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.533+0000 7f7e9359e640 1 -- 172.21.15.163:0/3395199891 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9415fca0 con 0x7f7e940d3830 2024-05-08T06:36:50.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.533+0000 7f7e92d9d640 1 --2- 172.21.15.163:0/3395199891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e941423c0 0x7f7e94142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:50.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.533+0000 7f7e9259c640 1 --2- 172.21.15.163:0/3395199891 >> v2:172.21.15.192:3300/0 conn(0x7f7e94163800 0x7f7e940d32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:50.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.534+0000 7f7e9359e640 1 --2- 172.21.15.163:0/3395199891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e940d3830 0x7f7e94144dd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7e940d3f90 tx=0x7f7e8c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d436a0754c3ecf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:50.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.534+0000 7f7e91d9b640 1 -- 172.21.15.163:0/3395199891 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c036070 con 0x7f7e940d3830 2024-05-08T06:36:50.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.534+0000 7f7e91d9b640 1 -- 172.21.15.163:0/3395199891 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8c031600 con 0x7f7e940d3830 2024-05-08T06:36:50.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.534+0000 7f7e91d9b640 1 -- 172.21.15.163:0/3395199891 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0319d0 con 0x7f7e940d3830 2024-05-08T06:36:50.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.534+0000 7f7e9919d640 1 -- 172.21.15.163:0/3395199891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e940d3830 msgr2=0x7f7e94144dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.534+0000 7f7e9919d640 1 --2- 172.21.15.163:0/3395199891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e940d3830 0x7f7e94144dd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7e940d3f90 tx=0x7f7e8c02a6a0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.534+0000 7f7e9919d640 1 -- 172.21.15.163:0/3395199891 shutdown_connections 2024-05-08T06:36:50.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.534+0000 7f7e9919d640 1 --2- 172.21.15.163:0/3395199891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e940d3830 0x7f7e94144dd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.534+0000 7f7e9919d640 1 --2- 172.21.15.163:0/3395199891 >> v2:172.21.15.192:3300/0 conn(0x7f7e94163800 0x7f7e940d32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.534+0000 7f7e9919d640 1 --2- 172.21.15.163:0/3395199891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e941423c0 0x7f7e94142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.534+0000 7f7e9919d640 1 -- 172.21.15.163:0/3395199891 >> 172.21.15.163:0/3395199891 conn(0x7f7e940682f0 msgr2=0x7f7e94068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:50.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.534+0000 7f7e9919d640 1 -- 172.21.15.163:0/3395199891 shutdown_connections 2024-05-08T06:36:50.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.534+0000 7f7e9919d640 1 -- 172.21.15.163:0/3395199891 wait complete. 2024-05-08T06:36:50.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.536+0000 7f7e9919d640 1 Processor -- start 2024-05-08T06:36:50.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.536+0000 7f7e9919d640 1 -- start start 2024-05-08T06:36:50.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9919d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940d3830 0x7f7e940b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9919d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e941423c0 0x7f7e940ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9919d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e94163800 0x7f7e9413e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9919d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94160540 con 0x7f7e941423c0 2024-05-08T06:36:50.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9919d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94160260 con 0x7f7e94163800 2024-05-08T06:36:50.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9919d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7e941630a0 con 0x7f7e940d3830 2024-05-08T06:36:50.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9259c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e941423c0 0x7f7e940ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9259c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e941423c0 0x7f7e940ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44334/0 (socket says 172.21.15.163:44334) 2024-05-08T06:36:50.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9259c640 1 -- 172.21.15.163:0/572002539 learned_addr learned my addr 172.21.15.163:0/572002539 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9359e640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e94163800 0x7f7e9413e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e92d9d640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940d3830 0x7f7e940b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9259c640 1 -- 172.21.15.163:0/572002539 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940d3830 msgr2=0x7f7e940b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9259c640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940d3830 0x7f7e940b2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9259c640 1 -- 172.21.15.163:0/572002539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e94163800 msgr2=0x7f7e9413e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9259c640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e94163800 0x7f7e9413e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9259c640 1 -- 172.21.15.163:0/572002539 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e940ce5c0 con 0x7f7e941423c0 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e92d9d640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940d3830 0x7f7e940b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9359e640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e94163800 0x7f7e9413e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9259c640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e941423c0 0x7f7e940ce080 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800d7d0 tx=0x7f7e8800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e7b7fe640 1 -- 172.21.15.163:0/572002539 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88004030 con 0x7f7e941423c0 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e7b7fe640 1 -- 172.21.15.163:0/572002539 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e880041d0 con 0x7f7e941423c0 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9919d640 1 -- 172.21.15.163:0/572002539 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e940b33d0 con 0x7f7e941423c0 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e7b7fe640 1 -- 172.21.15.163:0/572002539 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8800b6c0 con 0x7f7e941423c0 2024-05-08T06:36:50.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.537+0000 7f7e9919d640 1 -- 172.21.15.163:0/572002539 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e940b3c70 con 0x7f7e941423c0 2024-05-08T06:36:50.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.538+0000 7f7e9919d640 1 -- 172.21.15.163:0/572002539 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e94075dc0 con 0x7f7e941423c0 2024-05-08T06:36:50.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.539+0000 7f7e7b7fe640 1 -- 172.21.15.163:0/572002539 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7e8800b860 con 0x7f7e941423c0 2024-05-08T06:36:50.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.539+0000 7f7e7b7fe640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7e800762d0 0x7f7e80078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.539+0000 7f7e92d9d640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7e800762d0 0x7f7e80078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.539+0000 7f7e7b7fe640 1 -- 172.21.15.163:0/572002539 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7e880977c0 con 0x7f7e941423c0 2024-05-08T06:36:50.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.540+0000 7f7e92d9d640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7e800762d0 0x7f7e80078790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e941f88e0 tx=0x7f7e84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:50.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.544+0000 7f7e7b7fe640 1 -- 172.21.15.163:0/572002539 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7e880611b0 con 0x7f7e941423c0 2024-05-08T06:36:50.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.774+0000 7f7e9919d640 1 -- 172.21.15.163:0/572002539 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7e94072da0 con 0x7f7e941423c0 2024-05-08T06:36:50.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.777+0000 7f7e7b7fe640 1 -- 172.21.15.163:0/572002539 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+189527 (secure 0 0 0) 0x7f7e88060ab0 con 0x7f7e941423c0 2024-05-08T06:36:50.779 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:50.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.780+0000 7f7e9919d640 1 -- 172.21.15.163:0/572002539 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7e800762d0 msgr2=0x7f7e80078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.780+0000 7f7e9919d640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7e800762d0 0x7f7e80078790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e941f88e0 tx=0x7f7e84008040 comp rx=0 tx=0).stop 2024-05-08T06:36:50.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.780+0000 7f7e9919d640 1 -- 172.21.15.163:0/572002539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e941423c0 msgr2=0x7f7e940ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.780+0000 7f7e9919d640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e941423c0 0x7f7e940ce080 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800d7d0 tx=0x7f7e8800dc90 comp rx=0 tx=0).stop 2024-05-08T06:36:50.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.780+0000 7f7e9919d640 1 -- 172.21.15.163:0/572002539 shutdown_connections 2024-05-08T06:36:50.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.780+0000 7f7e9919d640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7e800762d0 0x7f7e80078790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.780+0000 7f7e9919d640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e94163800 0x7f7e9413e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.780+0000 7f7e9919d640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e941423c0 0x7f7e940ce080 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.780+0000 7f7e9919d640 1 --2- 172.21.15.163:0/572002539 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940d3830 0x7f7e940b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.780+0000 7f7e9919d640 1 -- 172.21.15.163:0/572002539 >> 172.21.15.163:0/572002539 conn(0x7f7e940682f0 msgr2=0x7f7e940b6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:50.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.780+0000 7f7e9919d640 1 -- 172.21.15.163:0/572002539 shutdown_connections 2024-05-08T06:36:50.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.780+0000 7f7e9919d640 1 -- 172.21.15.163:0/572002539 wait complete. 2024-05-08T06:36:50.790 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2781435851},{"type":"v1","addr":"172.21.15.163:6833","nonce":2781435851}]},"active_addr":"172.21.15.163:6833/2781435851","active_change":"2024-05-08T06:36:22.088079+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":2803609994}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":3673985592}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":559302364}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":1473921085}]}]} 2024-05-08T06:36:50.795 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-08T06:36:50.795 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T06:36:50.795 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:36:50.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.896+0000 7f437ea45640 1 Processor -- start 2024-05-08T06:36:50.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.896+0000 7f437ea45640 1 -- start start 2024-05-08T06:36:50.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f437ea45640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43780b0810 0x7f43780b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f437ea45640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4378070ea0 0x7f43780b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f437ea45640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43780741d0 con 0x7f43780b0430 2024-05-08T06:36:50.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f437ea45640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f43780b1900 con 0x7f4378070ea0 2024-05-08T06:36:50.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f437ea45640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f43780b5610 con 0x7f43780b0810 2024-05-08T06:36:50.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f4377fff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f43780b0430 0x7f437815f9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57020/0 (socket says 172.21.15.163:57020) 2024-05-08T06:36:50.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f4377fff640 1 -- 172.21.15.163:0/4038384742 learned_addr learned my addr 172.21.15.163:0/4038384742 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:50.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f437cfe2640 1 --2- 172.21.15.163:0/4038384742 >> v2:172.21.15.192:3300/0 conn(0x7f4378070ea0 0x7f43780b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f436ffff640 1 --2- 172.21.15.163:0/4038384742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43780b0810 0x7f43780b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f43777fe640 1 -- 172.21.15.163:0/4038384742 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1020812840 0 0) 0x7f43780741d0 con 0x7f43780b0430 2024-05-08T06:36:50.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f43777fe640 1 -- 172.21.15.163:0/4038384742 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4364003520 con 0x7f43780b0430 2024-05-08T06:36:50.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f43777fe640 1 -- 172.21.15.163:0/4038384742 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4265283754 0 0) 0x7f4364003520 con 0x7f43780b0430 2024-05-08T06:36:50.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f436ffff640 1 -- 172.21.15.163:0/4038384742 >> v2:172.21.15.192:3300/0 conn(0x7f4378070ea0 msgr2=0x7f43780b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:50.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f436ffff640 1 --2- 172.21.15.163:0/4038384742 >> v2:172.21.15.192:3300/0 conn(0x7f4378070ea0 0x7f43780b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f436ffff640 1 -- 172.21.15.163:0/4038384742 >> v1:172.21.15.163:6789/0 conn(0x7f43780b0430 legacy=0x7f437815f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f436ffff640 1 -- 172.21.15.163:0/4038384742 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43780b6fd0 con 0x7f43780b0810 2024-05-08T06:36:50.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.897+0000 7f437cfe2640 1 --2- 172.21.15.163:0/4038384742 >> v2:172.21.15.192:3300/0 conn(0x7f4378070ea0 0x7f43780b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:50.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.898+0000 7f436ffff640 1 --2- 172.21.15.163:0/4038384742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43780b0810 0x7f43780b4fa0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43600097a0 tx=0x7f436002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e562ed536e0e1211 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:50.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.898+0000 7f43777fe640 1 -- 172.21.15.163:0/4038384742 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360037070 con 0x7f43780b0810 2024-05-08T06:36:50.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.898+0000 7f43777fe640 1 -- 172.21.15.163:0/4038384742 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f436002ab10 con 0x7f43780b0810 2024-05-08T06:36:50.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.898+0000 7f43777fe640 1 -- 172.21.15.163:0/4038384742 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680016f0 con 0x7f43780b0810 2024-05-08T06:36:50.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.899+0000 7f437ea45640 1 -- 172.21.15.163:0/4038384742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43780b0810 msgr2=0x7f43780b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.899+0000 7f437ea45640 1 --2- 172.21.15.163:0/4038384742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43780b0810 0x7f43780b4fa0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43600097a0 tx=0x7f436002a080 comp rx=0 tx=0).stop 2024-05-08T06:36:50.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.899+0000 7f437ea45640 1 -- 172.21.15.163:0/4038384742 shutdown_connections 2024-05-08T06:36:50.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.899+0000 7f437ea45640 1 --2- 172.21.15.163:0/4038384742 >> v2:172.21.15.192:3300/0 conn(0x7f4378070ea0 0x7f43780b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.899+0000 7f437ea45640 1 --2- 172.21.15.163:0/4038384742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43780b0810 0x7f43780b4fa0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.899+0000 7f437ea45640 1 -- 172.21.15.163:0/4038384742 >> 172.21.15.163:0/4038384742 conn(0x7f437806d430 msgr2=0x7f437806d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:50.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.899+0000 7f437ea45640 1 -- 172.21.15.163:0/4038384742 shutdown_connections 2024-05-08T06:36:50.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.899+0000 7f437ea45640 1 -- 172.21.15.163:0/4038384742 wait complete. 2024-05-08T06:36:50.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.900+0000 7f437ea45640 1 Processor -- start 2024-05-08T06:36:50.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.900+0000 7f437ea45640 1 -- start start 2024-05-08T06:36:50.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f437ea45640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4378070ea0 0x7f43780d0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f437ea45640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43780b0430 0x7f437814aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f437ea45640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43780b0810 0x7f43780cea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f437ea45640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f43780b6db0 con 0x7f43780b0430 2024-05-08T06:36:50.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f437ea45640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f43780b6c30 con 0x7f4378070ea0 2024-05-08T06:36:50.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f437ea45640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f43780b4170 con 0x7f43780b0810 2024-05-08T06:36:50.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f436ffff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43780b0430 0x7f437814aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f436ffff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43780b0430 0x7f437814aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44342/0 (socket says 172.21.15.163:44342) 2024-05-08T06:36:50.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f436ffff640 1 -- 172.21.15.163:0/3897162253 learned_addr learned my addr 172.21.15.163:0/3897162253 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:50.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f4377fff640 1 --2- 172.21.15.163:0/3897162253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4378070ea0 0x7f43780d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f437cfe2640 1 --2- 172.21.15.163:0/3897162253 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43780b0810 0x7f43780cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f436ffff640 1 -- 172.21.15.163:0/3897162253 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43780b0810 msgr2=0x7f43780cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f436ffff640 1 --2- 172.21.15.163:0/3897162253 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43780b0810 0x7f43780cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f436ffff640 1 -- 172.21.15.163:0/3897162253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4378070ea0 msgr2=0x7f43780d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:50.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f436ffff640 1 --2- 172.21.15.163:0/3897162253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4378070ea0 0x7f43780d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:50.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f436ffff640 1 -- 172.21.15.163:0/3897162253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43780d1770 con 0x7f43780b0430 2024-05-08T06:36:50.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f436ffff640 1 --2- 172.21.15.163:0/3897162253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43780b0430 0x7f437814aa80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f43600098c0 tx=0x7f43600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:50.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.901+0000 7f43757fa640 1 -- 172.21.15.163:0/3897162253 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360037050 con 0x7f43780b0430 2024-05-08T06:36:50.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.902+0000 7f43757fa640 1 -- 172.21.15.163:0/3897162253 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4360043070 con 0x7f43780b0430 2024-05-08T06:36:50.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.902+0000 7f437ea45640 1 -- 172.21.15.163:0/3897162253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43780d0bf0 con 0x7f43780b0430 2024-05-08T06:36:50.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.902+0000 7f43757fa640 1 -- 172.21.15.163:0/3897162253 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360006e10 con 0x7f43780b0430 2024-05-08T06:36:50.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.902+0000 7f437ea45640 1 -- 172.21.15.163:0/3897162253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43780d1490 con 0x7f43780b0430 2024-05-08T06:36:50.905 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.903+0000 7f437ea45640 1 -- 172.21.15.163:0/3897162253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43780b0c20 con 0x7f43780b0430 2024-05-08T06:36:50.905 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.903+0000 7f43757fa640 1 -- 172.21.15.163:0/3897162253 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f436003ca10 con 0x7f43780b0430 2024-05-08T06:36:50.905 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.904+0000 7f43757fa640 1 --2- 172.21.15.163:0/3897162253 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f43580762d0 0x7f4358078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:50.905 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.904+0000 7f4377fff640 1 --2- 172.21.15.163:0/3897162253 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f43580762d0 0x7f4358078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:50.905 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.904+0000 7f43757fa640 1 -- 172.21.15.163:0/3897162253 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f43600b7ec0 con 0x7f43780b0430 2024-05-08T06:36:50.905 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.904+0000 7f4377fff640 1 --2- 172.21.15.163:0/3897162253 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f43580762d0 0x7f4358078790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4368005880 tx=0x7f4368001db0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:50.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:50.908+0000 7f43757fa640 1 -- 172.21.15.163:0/3897162253 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f43780b0c20 con 0x7f43780b0430 2024-05-08T06:36:51.087 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.088+0000 7f437ea45640 1 -- 172.21.15.163:0/3897162253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f437806a550 con 0x7f43780b0430 2024-05-08T06:36:51.088 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.089+0000 7f43757fa640 1 -- 172.21.15.163:0/3897162253 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f4360081260 con 0x7f43780b0430 2024-05-08T06:36:51.089 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:51.089 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:36:42.641260+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:36:38.853461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:36:39.952926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:36:51.091 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.091+0000 7f437ea45640 1 -- 172.21.15.163:0/3897162253 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f43580762d0 msgr2=0x7f4358078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.091 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.091+0000 7f437ea45640 1 --2- 172.21.15.163:0/3897162253 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f43580762d0 0x7f4358078790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4368005880 tx=0x7f4368001db0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.091 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.091+0000 7f437ea45640 1 -- 172.21.15.163:0/3897162253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43780b0430 msgr2=0x7f437814aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.091 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.091+0000 7f437ea45640 1 --2- 172.21.15.163:0/3897162253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43780b0430 0x7f437814aa80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f43600098c0 tx=0x7f43600046c0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.091 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.091+0000 7f437ea45640 1 -- 172.21.15.163:0/3897162253 shutdown_connections 2024-05-08T06:36:51.091 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.091+0000 7f437ea45640 1 --2- 172.21.15.163:0/3897162253 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f43580762d0 0x7f4358078790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.091 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.091+0000 7f437ea45640 1 --2- 172.21.15.163:0/3897162253 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43780b0810 0x7f43780cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.091 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.091+0000 7f437ea45640 1 --2- 172.21.15.163:0/3897162253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43780b0430 0x7f437814aa80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.092 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.091+0000 7f437ea45640 1 --2- 172.21.15.163:0/3897162253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4378070ea0 0x7f43780d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.092 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.092+0000 7f437ea45640 1 -- 172.21.15.163:0/3897162253 >> 172.21.15.163:0/3897162253 conn(0x7f437806d430 msgr2=0x7f437815ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.092 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.092+0000 7f437ea45640 1 -- 172.21.15.163:0/3897162253 shutdown_connections 2024-05-08T06:36:51.092 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.092+0000 7f437ea45640 1 -- 172.21.15.163:0/3897162253 wait complete. 2024-05-08T06:36:51.102 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T06:36:51.103 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:36:51.202 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.203+0000 7f2eb795f640 1 Processor -- start 2024-05-08T06:36:51.206 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.203+0000 7f2eb795f640 1 -- start start 2024-05-08T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.204+0000 7f2eb795f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2eb0162900 0x7f2eb0073f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.204+0000 7f2eb795f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2eb00d2170 0x7f2eb00d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.204+0000 7f2eb795f640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eb00b74f0 con 0x7f2eb00aef80 2024-05-08T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.204+0000 7f2eb795f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2eb00d2b00 con 0x7f2eb0162900 2024-05-08T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.204+0000 7f2eb795f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb0162ce0 con 0x7f2eb00d2170 2024-05-08T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.204+0000 7f2eb56fb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2eb0162900 0x7f2eb0073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.204+0000 7f2eb5efc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2eb00d2170 0x7f2eb00d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.204+0000 7f2eb56fb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2eb0162900 0x7f2eb0073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35580/0 (socket says 172.21.15.163:35580) 2024-05-08T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.204+0000 7f2eb56fb640 1 -- 172.21.15.163:0/2419143332 learned_addr learned my addr 172.21.15.163:0/2419143332 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.204+0000 7f2eb5efc640 1 --2- 172.21.15.163:0/2419143332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2eb00d2170 0x7f2eb00d25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:48106/0 (socket says 172.21.15.163:48106) 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.204+0000 7f2eb56fb640 1 -- 172.21.15.163:0/2419143332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2eb00d2170 msgr2=0x7f2eb00d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.204+0000 7f2eb56fb640 1 --2- 172.21.15.163:0/2419143332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2eb00d2170 0x7f2eb00d25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.204+0000 7f2eb56fb640 1 -- 172.21.15.163:0/2419143332 >> v1:172.21.15.192:6789/0 conn(0x7f2eb00aef80 legacy=0x7f2eb00b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.204+0000 7f2eb56fb640 1 -- 172.21.15.163:0/2419143332 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb00b50b0 con 0x7f2eb0162900 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.205+0000 7f2eb56fb640 1 --2- 172.21.15.163:0/2419143332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eb0162900 0x7f2eb0073f70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009870 tx=0x7f2eac02a1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e634ff9f2a23ff3b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.205+0000 7f2ea3fff640 1 -- 172.21.15.163:0/2419143332 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac037070 con 0x7f2eb0162900 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.205+0000 7f2ea3fff640 1 -- 172.21.15.163:0/2419143332 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eac032440 con 0x7f2eb0162900 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.205+0000 7f2ea3fff640 1 -- 172.21.15.163:0/2419143332 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac032810 con 0x7f2eb0162900 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.206+0000 7f2eb795f640 1 -- 172.21.15.163:0/2419143332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eb0162900 msgr2=0x7f2eb0073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.206+0000 7f2eb795f640 1 --2- 172.21.15.163:0/2419143332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eb0162900 0x7f2eb0073f70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009870 tx=0x7f2eac02a1b0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.206+0000 7f2eb795f640 1 -- 172.21.15.163:0/2419143332 shutdown_connections 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.206+0000 7f2eb795f640 1 --2- 172.21.15.163:0/2419143332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2eb00d2170 0x7f2eb00d25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.206+0000 7f2eb795f640 1 --2- 172.21.15.163:0/2419143332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eb0162900 0x7f2eb0073f70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.206+0000 7f2eb795f640 1 -- 172.21.15.163:0/2419143332 >> 172.21.15.163:0/2419143332 conn(0x7f2eb006d610 msgr2=0x7f2eb006d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.206+0000 7f2eb795f640 1 -- 172.21.15.163:0/2419143332 shutdown_connections 2024-05-08T06:36:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.206+0000 7f2eb795f640 1 -- 172.21.15.163:0/2419143332 wait complete. 2024-05-08T06:36:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.207+0000 7f2eb795f640 1 Processor -- start 2024-05-08T06:36:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.207+0000 7f2eb795f640 1 -- start start 2024-05-08T06:36:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb795f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2eb00aef80 0x7f2eb0157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb795f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eb00d2170 0x7f2eb0151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb795f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2eb0162900 0x7f2eb0152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb795f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb006f4d0 con 0x7f2eb00aef80 2024-05-08T06:36:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb795f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb00b5670 con 0x7f2eb00d2170 2024-05-08T06:36:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb795f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb006f9f0 con 0x7f2eb0162900 2024-05-08T06:36:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb56fb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2eb00aef80 0x7f2eb0157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb56fb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2eb00aef80 0x7f2eb0157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44352/0 (socket says 172.21.15.163:44352) 2024-05-08T06:36:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb56fb640 1 -- 172.21.15.163:0/2793050726 learned_addr learned my addr 172.21.15.163:0/2793050726 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb5efc640 1 --2- 172.21.15.163:0/2793050726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2eb0162900 0x7f2eb0152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb4efa640 1 --2- 172.21.15.163:0/2793050726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eb00d2170 0x7f2eb0151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb56fb640 1 -- 172.21.15.163:0/2793050726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2eb0162900 msgr2=0x7f2eb0152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb56fb640 1 --2- 172.21.15.163:0/2793050726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2eb0162900 0x7f2eb0152020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb56fb640 1 -- 172.21.15.163:0/2793050726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eb00d2170 msgr2=0x7f2eb0151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb56fb640 1 --2- 172.21.15.163:0/2793050726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eb00d2170 0x7f2eb0151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb56fb640 1 -- 172.21.15.163:0/2793050726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb0154170 con 0x7f2eb00aef80 2024-05-08T06:36:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb56fb640 1 --2- 172.21.15.163:0/2793050726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2eb00aef80 0x7f2eb0157df0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2eac000c00 tx=0x7f2eac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2ea1ffb640 1 -- 172.21.15.163:0/2793050726 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac037040 con 0x7f2eb00aef80 2024-05-08T06:36:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2ea1ffb640 1 -- 172.21.15.163:0/2793050726 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eac042070 con 0x7f2eb00aef80 2024-05-08T06:36:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2ea1ffb640 1 -- 172.21.15.163:0/2793050726 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac030880 con 0x7f2eb00aef80 2024-05-08T06:36:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.208+0000 7f2eb795f640 1 -- 172.21.15.163:0/2793050726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb0154450 con 0x7f2eb00aef80 2024-05-08T06:36:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.209+0000 7f2eb795f640 1 -- 172.21.15.163:0/2793050726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb0154a10 con 0x7f2eb00aef80 2024-05-08T06:36:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.209+0000 7f2eb795f640 1 -- 172.21.15.163:0/2793050726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb006a550 con 0x7f2eb00aef80 2024-05-08T06:36:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.210+0000 7f2ea1ffb640 1 -- 172.21.15.163:0/2793050726 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2eac030a20 con 0x7f2eb00aef80 2024-05-08T06:36:51.215 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.211+0000 7f2ea1ffb640 1 --2- 172.21.15.163:0/2793050726 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2e94076300 0x7f2e940787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.215 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.211+0000 7f2eb4efa640 1 --2- 172.21.15.163:0/2793050726 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2e94076300 0x7f2e940787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.215 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.211+0000 7f2ea1ffb640 1 -- 172.21.15.163:0/2793050726 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2eac0b7e50 con 0x7f2eb00aef80 2024-05-08T06:36:51.215 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.211+0000 7f2eb4efa640 1 --2- 172.21.15.163:0/2793050726 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2e94076300 0x7f2e940787c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2eb01622a0 tx=0x7f2ea4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.215 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.215+0000 7f2ea1ffb640 1 -- 172.21.15.163:0/2793050726 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2eac0817b0 con 0x7f2eb00aef80 2024-05-08T06:36:51.398 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.396+0000 7f2eb795f640 1 -- 172.21.15.163:0/2793050726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2eb01603c0 con 0x7f2eb00aef80 2024-05-08T06:36:51.398 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.397+0000 7f2ea1ffb640 1 -- 172.21.15.163:0/2793050726 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f2eac0810b0 con 0x7f2eb00aef80 2024-05-08T06:36:51.399 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:51.399 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:36:42.641260+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:36:38.853461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:36:39.952926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:36:51.400 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.399+0000 7f2eb795f640 1 -- 172.21.15.163:0/2793050726 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2e94076300 msgr2=0x7f2e940787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.400 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.399+0000 7f2eb795f640 1 --2- 172.21.15.163:0/2793050726 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2e94076300 0x7f2e940787c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2eb01622a0 tx=0x7f2ea4009040 comp rx=0 tx=0).stop 2024-05-08T06:36:51.400 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.399+0000 7f2eb795f640 1 -- 172.21.15.163:0/2793050726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2eb00aef80 msgr2=0x7f2eb0157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.400 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.399+0000 7f2eb795f640 1 --2- 172.21.15.163:0/2793050726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2eb00aef80 0x7f2eb0157df0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2eac000c00 tx=0x7f2eac0046c0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.400 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.399+0000 7f2eb795f640 1 -- 172.21.15.163:0/2793050726 shutdown_connections 2024-05-08T06:36:51.400 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.399+0000 7f2eb795f640 1 --2- 172.21.15.163:0/2793050726 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2e94076300 0x7f2e940787c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.399+0000 7f2eb795f640 1 --2- 172.21.15.163:0/2793050726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2eb0162900 0x7f2eb0152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.399+0000 7f2eb795f640 1 --2- 172.21.15.163:0/2793050726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eb00d2170 0x7f2eb0151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.399+0000 7f2eb795f640 1 --2- 172.21.15.163:0/2793050726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2eb00aef80 0x7f2eb0157df0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.399+0000 7f2eb795f640 1 -- 172.21.15.163:0/2793050726 >> 172.21.15.163:0/2793050726 conn(0x7f2eb006d610 msgr2=0x7f2eb00739c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.399+0000 7f2eb795f640 1 -- 172.21.15.163:0/2793050726 shutdown_connections 2024-05-08T06:36:51.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.399+0000 7f2eb795f640 1 -- 172.21.15.163:0/2793050726 wait complete. 2024-05-08T06:36:51.410 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T06:36:51.411 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T06:36:51.411 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T06:36:51.412 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T06:36:51.412 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T06:36:51.413 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T06:36:51.413 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-08T06:36:51.413 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-08T06:36:51.585 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.585+0000 7f16a3210640 1 Processor -- start 2024-05-08T06:36:51.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.586+0000 7f5aa98c2640 1 Processor -- start 2024-05-08T06:36:51.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.586+0000 7f5aa98c2640 1 -- start start 2024-05-08T06:36:51.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.586+0000 7f16a3210640 1 -- start start 2024-05-08T06:36:51.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.586+0000 7f83ebfff640 1 Processor -- start 2024-05-08T06:36:51.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.587+0000 7f83ebfff640 1 -- start start 2024-05-08T06:36:51.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.587+0000 7f16a3210640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f169c1629e0 0x7f169c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.587+0000 7fb5b1c1d640 1 Processor -- start 2024-05-08T06:36:51.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.587+0000 7f16a3210640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f169c0d2030 0x7f169c0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.587+0000 7f16a3210640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f169c0d38f0 con 0x7f169c0b55d0 2024-05-08T06:36:51.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.587+0000 7f16a3210640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f169c0d2470 con 0x7f169c0d2030 2024-05-08T06:36:51.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.587+0000 7f16a3210640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f169c0b42d0 con 0x7f169c1629e0 2024-05-08T06:36:51.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.587+0000 7fb5b1c1d640 1 -- start start 2024-05-08T06:36:51.588 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.587+0000 7f78dbb0a640 1 Processor -- start 2024-05-08T06:36:51.589 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.587+0000 7fe1553c1640 1 Processor -- start 2024-05-08T06:36:51.589 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f5aa98c2640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5a9c0e32c0 0x7f5a9c053820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.589 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f5aa98c2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a9c0550a0 0x7f5a9c006a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.589 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f5aa98c2640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a9c053d60 con 0x7f5a9c03b810 2024-05-08T06:36:51.589 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f5aa98c2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f5a9c054000 con 0x7f5a9c0550a0 2024-05-08T06:36:51.589 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f5aa98c2640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5a9c0542e0 con 0x7f5a9c0e32c0 2024-05-08T06:36:51.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f78dbb0a640 1 -- start start 2024-05-08T06:36:51.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f5aa90c1640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f5a9c03b810 0x7f5a9c0e6e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:57258/0 (socket says 172.21.15.163:57258) 2024-05-08T06:36:51.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f5aa90c1640 1 -- 172.21.15.163:0/2641489395 learned_addr learned my addr 172.21.15.163:0/2641489395 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:51.591 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f83ebfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f83e4008c20 0x7f83e4007dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.591 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f83ebfff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f83e40e40e0 0x7f83e4056040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.591 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f83ebfff640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83e40e4770 con 0x7f83e4008840 2024-05-08T06:36:51.591 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f83ebfff640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f83e4055bb0 con 0x7f83e40e40e0 2024-05-08T06:36:51.591 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f83ebfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f83e4055280 con 0x7f83e4008c20 2024-05-08T06:36:51.591 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f83eaffd640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f83e4008840 0x7f83e40e4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57026/0 (socket says 172.21.15.163:57026) 2024-05-08T06:36:51.591 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f83eaffd640 1 -- 172.21.15.163:0/255631172 learned_addr learned my addr 172.21.15.163:0/255631172 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:51.591 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7fe1553c1640 1 -- start start 2024-05-08T06:36:51.592 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f5aa37fe640 1 -- 172.21.15.163:0/2641489395 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3781452722 0 0) 0x7f5a9c053d60 con 0x7f5a9c03b810 2024-05-08T06:36:51.592 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f83eb7fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f83e40e40e0 0x7f83e4056040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.592 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f5aa88c0640 1 --2- 172.21.15.163:0/2641489395 >> v2:172.21.15.192:3300/0 conn(0x7f5a9c0e32c0 0x7f5a9c053820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f5aa3fff640 1 --2- 172.21.15.163:0/2641489395 >> v2:172.21.15.163:3300/0 conn(0x7f5a9c0550a0 0x7f5a9c006a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f5aa37fe640 1 -- 172.21.15.163:0/2641489395 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a94003520 con 0x7f5a9c03b810 2024-05-08T06:36:51.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7fb5b1c1d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ac15f4c0 0x7fb5ac163020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f16a220e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f169c1629e0 0x7f169c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f16a220e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f169c1629e0 0x7f169c0d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:48134/0 (socket says 172.21.15.163:48134) 2024-05-08T06:36:51.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f16a220e640 1 -- 172.21.15.163:0/4161724585 learned_addr learned my addr 172.21.15.163:0/4161724585 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.588+0000 7f16a1a0d640 1 --2- 172.21.15.163:0/4161724585 >> v2:172.21.15.192:3300/0 conn(0x7f169c0d2030 0x7f169c0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f16a220e640 1 -- 172.21.15.163:0/4161724585 >> v2:172.21.15.192:3300/0 conn(0x7f169c0d2030 msgr2=0x7f169c0b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f16a220e640 1 --2- 172.21.15.163:0/4161724585 >> v2:172.21.15.192:3300/0 conn(0x7f169c0d2030 0x7f169c0b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f16a220e640 1 -- 172.21.15.163:0/4161724585 >> v1:172.21.15.192:6789/0 conn(0x7f169c0b55d0 legacy=0x7f169c0b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f16a220e640 1 -- 172.21.15.163:0/4161724585 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169c0d3bd0 con 0x7f169c1629e0 2024-05-08T06:36:51.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f5aa37fe640 1 -- 172.21.15.163:0/2641489395 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1269063019 0 0) 0x7f5a94003520 con 0x7f5a9c03b810 2024-05-08T06:36:51.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f5aa37fe640 1 -- 172.21.15.163:0/2641489395 >> v2:172.21.15.192:3300/0 conn(0x7f5a9c0e32c0 msgr2=0x7f5a9c053820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f5aa37fe640 1 --2- 172.21.15.163:0/2641489395 >> v2:172.21.15.192:3300/0 conn(0x7f5a9c0e32c0 0x7f5a9c053820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f5aa37fe640 1 -- 172.21.15.163:0/2641489395 >> v2:172.21.15.163:3300/0 conn(0x7f5a9c0550a0 msgr2=0x7f5a9c006a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f970e41d640 1 Processor -- start 2024-05-08T06:36:51.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f78dbb0a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78cc048b00 0x7f78cc0526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f78dbb0a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78cc03a320 con 0x7f78cc0eaca0 2024-05-08T06:36:51.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f78dbb0a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78cc03a600 con 0x7f78cc03b0e0 2024-05-08T06:36:51.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f78dbb0a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f78cc052bf0 con 0x7f78cc048b00 2024-05-08T06:36:51.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f78d98a6640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f78cc0eaca0 0x7f78cc0483f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57032/0 (socket says 172.21.15.163:57032) 2024-05-08T06:36:51.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f78d98a6640 1 -- 172.21.15.163:0/1847367010 learned_addr learned my addr 172.21.15.163:0/1847367010 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:51.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7fe1553c1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe1500bab60 0x7fe1500d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7fe1553c1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1501637e0 con 0x7fe15015f4c0 2024-05-08T06:36:51.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7fe1553c1640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe150163ac0 con 0x7fe1500d0550 2024-05-08T06:36:51.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7fe1553c1640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fe150163da0 con 0x7fe1500bab60 2024-05-08T06:36:51.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f970e41d640 1 -- start start 2024-05-08T06:36:51.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f78d90a5640 1 --2- 172.21.15.163:0/1847367010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78cc048b00 0x7f78cc0526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7fe14e7fc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe1500bab60 0x7fe1500d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f5aa37fe640 1 --2- 172.21.15.163:0/2641489395 >> v2:172.21.15.163:3300/0 conn(0x7f5a9c0550a0 0x7f5a9c006a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f5aa37fe640 1 -- 172.21.15.163:0/2641489395 --> v1:172.21.15.192:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9c0545c0 con 0x7f5a9c03b810 2024-05-08T06:36:51.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7fb5b1c1d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb5ac0d0720 0x7fb5ac0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7fe14e7fc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe1500bab60 0x7fe1500d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44386/0 (socket says 172.21.15.163:44386) 2024-05-08T06:36:51.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7fe14e7fc640 1 -- 172.21.15.163:0/1109463883 learned_addr learned my addr 172.21.15.163:0/1109463883 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f5aa37fe640 1 -- 172.21.15.163:0/2641489395 <== mon.1 v1:172.21.15.192:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f5a900016f0 con 0x7f5a9c03b810 2024-05-08T06:36:51.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f5aa37fe640 1 -- 172.21.15.163:0/2641489395 >> v1:172.21.15.192:6789/0 conn(0x7f5a9c03b810 legacy=0x7f5a9c0e6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f78d88a4640 1 -- 172.21.15.163:0/1847367010 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3685229848 0 0) 0x7f78cc03a600 con 0x7f78cc03b0e0 2024-05-08T06:36:51.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7fb5b1c1d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5ac163560 con 0x7fb5ac0bab60 2024-05-08T06:36:51.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7fb5b1c1d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb5ac1637e0 con 0x7fb5ac0d0720 2024-05-08T06:36:51.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7fb5b1c1d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ac163ac0 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7fb5ab7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ac15f4c0 0x7fb5ac163020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7fb5ab7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ac15f4c0 0x7fb5ac163020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:48144/0 (socket says 172.21.15.163:48144) 2024-05-08T06:36:51.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7fb5ab7fe640 1 -- 172.21.15.163:0/102044214 learned_addr learned my addr 172.21.15.163:0/102044214 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f5aa37fe640 1 --2- 172.21.15.163:0/2641489395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a94003a50 0x7f5a94003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f83eb7fe640 1 -- 172.21.15.163:0/255631172 >> v2:172.21.15.192:3300/0 conn(0x7f83e4008c20 msgr2=0x7f83e4007dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:36:51.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f83eb7fe640 1 --2- 172.21.15.163:0/255631172 >> v2:172.21.15.192:3300/0 conn(0x7f83e4008c20 0x7f83e4007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f83eb7fe640 1 -- 172.21.15.163:0/255631172 >> v1:172.21.15.163:6789/0 conn(0x7f83e4008840 legacy=0x7f83e40e4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.589+0000 7f83eb7fe640 1 -- 172.21.15.163:0/255631172 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e4055560 con 0x7f83e40e40e0 2024-05-08T06:36:51.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7fb5abfff640 1 --2- 172.21.15.163:0/102044214 >> v2:172.21.15.192:3300/0 conn(0x7fb5ac0d0720 0x7fb5ac0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f5aa37fe640 1 --2- 172.21.15.163:0/2641489395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a94004800 0x7f5a94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f5aa37fe640 1 --2- 172.21.15.163:0/2641489395 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a940052e0 0x7f5a9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f81b6eda640 1 Processor -- start 2024-05-08T06:36:51.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f78d88a4640 1 -- 172.21.15.163:0/1847367010 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78c0003520 con 0x7f78cc03b0e0 2024-05-08T06:36:51.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f5aa37fe640 1 -- 172.21.15.163:0/2641489395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c054000 con 0x7f5a94003a50 2024-05-08T06:36:51.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f5aa37fe640 1 -- 172.21.15.163:0/2641489395 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c0542e0 con 0x7f5a94004800 2024-05-08T06:36:51.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f5aa37fe640 1 -- 172.21.15.163:0/2641489395 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5a9400c280 con 0x7f5a940052e0 2024-05-08T06:36:51.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f5aa3fff640 1 --2- 172.21.15.163:0/2641489395 >> v2:172.21.15.163:3300/0 conn(0x7f5a9c0550a0 0x7f5a9c006a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:51.607 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7fb5ab7fe640 1 -- 172.21.15.163:0/102044214 >> v2:172.21.15.192:3300/0 conn(0x7fb5ac0d0720 msgr2=0x7fb5ac0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.607 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7fb5ab7fe640 1 --2- 172.21.15.163:0/102044214 >> v2:172.21.15.192:3300/0 conn(0x7fb5ac0d0720 0x7fb5ac0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.607 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7fb5ab7fe640 1 -- 172.21.15.163:0/102044214 >> v1:172.21.15.192:6789/0 conn(0x7fb5ac0bab60 legacy=0x7fb5ac0d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.607 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f16a220e640 1 --2- 172.21.15.163:0/4161724585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f169c1629e0 0x7f169c0d1af0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1698009870 tx=0x7f169802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcd2f39587a2b897 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.607 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f83eb7fe640 1 --2- 172.21.15.163:0/255631172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83e40e40e0 0x7f83e4056040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0514f0 tx=0x7f83ec0a6d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c3bd611e7fb6d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.608 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7fe14e7fc640 1 -- 172.21.15.163:0/1109463883 >> v1:172.21.15.192:6789/0 conn(0x7fe1500d0550 legacy=0x7fe1500d3cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T06:36:51.608 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f81b6eda640 1 -- start start 2024-05-08T06:36:51.608 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7fb5ab7fe640 1 -- 172.21.15.163:0/102044214 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ac163da0 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.608 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f5aa88c0640 1 --2- 172.21.15.163:0/2641489395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a94003a50 0x7f5a94003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f5aa90c1640 1 --2- 172.21.15.163:0/2641489395 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a940052e0 0x7f5a9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f78d88a4640 1 -- 172.21.15.163:0/1847367010 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1969657648 0 0) 0x7f78c0003520 con 0x7f78cc03b0e0 2024-05-08T06:36:51.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f78d90a5640 1 -- 172.21.15.163:0/1847367010 >> v1:172.21.15.192:6789/0 conn(0x7f78cc03b0e0 legacy=0x7f78cc0537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f78d90a5640 1 -- 172.21.15.163:0/1847367010 >> v1:172.21.15.163:6789/0 conn(0x7f78cc0eaca0 legacy=0x7f78cc0483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.590+0000 7f78d90a5640 1 -- 172.21.15.163:0/1847367010 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc03a8e0 con 0x7f78cc048b00 2024-05-08T06:36:51.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f16a120c640 1 -- 172.21.15.163:0/4161724585 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1698037070 con 0x7f169c1629e0 2024-05-08T06:36:51.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f970e41d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f97080b4630 0x7f97080cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f970e41d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9708073cb0 0x7f9708074120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f970e41d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9708072c10 con 0x7f97080cdbd0 2024-05-08T06:36:51.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f970e41d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f97080729c0 con 0x7f97080b4630 2024-05-08T06:36:51.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f970e41d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9708072ef0 con 0x7f9708073cb0 2024-05-08T06:36:51.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f16a120c640 1 -- 172.21.15.163:0/4161724585 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1698032440 con 0x7f169c1629e0 2024-05-08T06:36:51.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f16a120c640 1 -- 172.21.15.163:0/4161724585 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1698032810 con 0x7f169c1629e0 2024-05-08T06:36:51.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f970c9ba640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9708073cb0 0x7f9708074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f83e9ffb640 1 -- 172.21.15.163:0/255631172 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0b4070 con 0x7f83e40e40e0 2024-05-08T06:36:51.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f97077fe640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f97080cdbd0 0x7f970806f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:57286/0 (socket says 172.21.15.163:57286) 2024-05-08T06:36:51.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f97077fe640 1 -- 172.21.15.163:0/2031864973 learned_addr learned my addr 172.21.15.163:0/2031864973 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:51.613 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f83e9ffb640 1 -- 172.21.15.163:0/255631172 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ec0a7b80 con 0x7f83e40e40e0 2024-05-08T06:36:51.613 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f83e9ffb640 1 -- 172.21.15.163:0/255631172 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0af440 con 0x7f83e40e40e0 2024-05-08T06:36:51.613 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7fe14e7fc640 1 -- 172.21.15.163:0/1109463883 >> v1:172.21.15.163:6789/0 conn(0x7fe15015f4c0 legacy=0x7fe150163020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.613 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7fe14e7fc640 1 -- 172.21.15.163:0/1109463883 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1500d37b0 con 0x7fe1500bab60 2024-05-08T06:36:51.613 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f5aa3fff640 1 --2- 172.21.15.163:0/2641489395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a94004800 0x7f5a94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.614 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f9707fff640 1 --2- 172.21.15.163:0/2031864973 >> v2:172.21.15.163:3300/0 conn(0x7f97080b4630 0x7f97080cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.614 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f5aa90c1640 1 -- 172.21.15.163:0/2641489395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a94004800 msgr2=0x7f5a94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.614 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f5aa90c1640 1 --2- 172.21.15.163:0/2641489395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a94004800 0x7f5a94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.615 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f5aa90c1640 1 -- 172.21.15.163:0/2641489395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a94003a50 msgr2=0x7f5a94003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.615 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f5aa90c1640 1 --2- 172.21.15.163:0/2641489395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a94003a50 0x7f5a94003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.615 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f5aa90c1640 1 -- 172.21.15.163:0/2641489395 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9c0545c0 con 0x7f5a940052e0 2024-05-08T06:36:51.615 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f5aa3fff640 1 --2- 172.21.15.163:0/2641489395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a94004800 0x7f5a94004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:51.615 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f78d90a5640 1 --2- 172.21.15.163:0/1847367010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78cc048b00 0x7f78cc0526b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009750 tx=0x7f78c402a010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c5f8c7478a70a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.615 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f9706ffd640 1 -- 172.21.15.163:0/2031864973 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2145494620 0 0) 0x7f9708072c10 con 0x7f97080cdbd0 2024-05-08T06:36:51.616 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f970c9ba640 1 -- 172.21.15.163:0/2031864973 >> v2:172.21.15.163:3300/0 conn(0x7f97080b4630 msgr2=0x7f97080cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.616 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f970c9ba640 1 --2- 172.21.15.163:0/2031864973 >> v2:172.21.15.163:3300/0 conn(0x7f97080b4630 0x7f97080cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.616 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f970c9ba640 1 -- 172.21.15.163:0/2031864973 >> v1:172.21.15.192:6789/0 conn(0x7f97080cdbd0 legacy=0x7f970806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.616 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.591+0000 7f970c9ba640 1 -- 172.21.15.163:0/2031864973 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97080731d0 con 0x7f9708073cb0 2024-05-08T06:36:51.616 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7fb5ab7fe640 1 --2- 172.21.15.163:0/102044214 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ac15f4c0 0x7fb5ac163020 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009870 tx=0x7fb59c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a270dfb4371c79d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f81b6eda640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81b00d1270 0x7f81b00d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f81b6eda640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f81b00b4340 0x7f81b00b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f81b6eda640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81b00d1b90 con 0x7f81b0072510 2024-05-08T06:36:51.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f81b6eda640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f81b0074420 con 0x7f81b00b4340 2024-05-08T06:36:51.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f81b6eda640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f81b006e550 con 0x7f81b00d1270 2024-05-08T06:36:51.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f78d88a4640 1 -- 172.21.15.163:0/1847367010 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4037070 con 0x7f78cc048b00 2024-05-08T06:36:51.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f78d88a4640 1 -- 172.21.15.163:0/1847367010 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c402ab20 con 0x7f78cc048b00 2024-05-08T06:36:51.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f78d88a4640 1 -- 172.21.15.163:0/1847367010 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d00016f0 con 0x7f78cc048b00 2024-05-08T06:36:51.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7fb5aa7fc640 1 -- 172.21.15.163:0/102044214 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c037070 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7fb5aa7fc640 1 -- 172.21.15.163:0/102044214 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59c032440 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7fe14e7fc640 1 --2- 172.21.15.163:0/1109463883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1500bab60 0x7fe1500d0010 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe14800b1a0 tx=0x7fe14802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4cdae1a039af8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f83ebfff640 1 -- 172.21.15.163:0/255631172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83e40e40e0 msgr2=0x7f83e4056040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f83ebfff640 1 --2- 172.21.15.163:0/255631172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83e40e40e0 0x7f83e4056040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0514f0 tx=0x7f83ec0a6d30 comp rx=0 tx=0).stop 2024-05-08T06:36:51.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f5aa90c1640 1 -- 172.21.15.163:0/2641489395 reap_dead start 2024-05-08T06:36:51.620 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7fb5aa7fc640 1 -- 172.21.15.163:0/102044214 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c032810 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.620 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f5aa90c1640 1 --2- 172.21.15.163:0/2641489395 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a940052e0 0x7f5a9400b9c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000c240 tx=0x7f5a94005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb98661eb6522f94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.620 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7fe14dffb640 1 -- 172.21.15.163:0/1109463883 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14802d070 con 0x7fe1500bab60 2024-05-08T06:36:51.620 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f16a3210640 1 -- 172.21.15.163:0/4161724585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f169c1629e0 msgr2=0x7f169c0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f16a3210640 1 --2- 172.21.15.163:0/4161724585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f169c1629e0 0x7f169c0d1af0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1698009870 tx=0x7f169802a130 comp rx=0 tx=0).stop 2024-05-08T06:36:51.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f83ebfff640 1 -- 172.21.15.163:0/255631172 shutdown_connections 2024-05-08T06:36:51.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f83ebfff640 1 --2- 172.21.15.163:0/255631172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83e40e40e0 0x7f83e4056040 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f83ebfff640 1 --2- 172.21.15.163:0/255631172 >> v2:172.21.15.192:3300/0 conn(0x7f83e4008c20 0x7f83e4007dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7fe14dffb640 1 -- 172.21.15.163:0/1109463883 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe148031430 con 0x7fe1500bab60 2024-05-08T06:36:51.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7fe14dffb640 1 -- 172.21.15.163:0/1109463883 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148031800 con 0x7fe1500bab60 2024-05-08T06:36:51.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f83ebfff640 1 -- 172.21.15.163:0/255631172 >> 172.21.15.163:0/255631172 conn(0x7f83e4020040 msgr2=0x7f83e4056d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f970c9ba640 1 --2- 172.21.15.163:0/2031864973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9708073cb0 0x7f9708074120 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9700009d90 tx=0x7f970002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce39f1e143c6fdb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f81b5477640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f81b00b4340 0x7f81b00b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f81b5477640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f81b00b4340 0x7f81b00b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44422/0 (socket says 172.21.15.163:44422) 2024-05-08T06:36:51.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f81b5477640 1 -- 172.21.15.163:0/3471879457 learned_addr learned my addr 172.21.15.163:0/3471879457 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f81b4c76640 1 --2- 172.21.15.163:0/3471879457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81b00d1270 0x7f81b00d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f5aa37fe640 1 -- 172.21.15.163:0/2641489395 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90007d70 con 0x7f5a940052e0 2024-05-08T06:36:51.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.592+0000 7f81b4c76640 1 -- 172.21.15.163:0/3471879457 >> v2:172.21.15.163:3300/0 conn(0x7f81b00b4340 msgr2=0x7f81b00b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f83ebfff640 1 -- 172.21.15.163:0/255631172 shutdown_connections 2024-05-08T06:36:51.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f16a3210640 1 -- 172.21.15.163:0/4161724585 shutdown_connections 2024-05-08T06:36:51.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f16a3210640 1 --2- 172.21.15.163:0/4161724585 >> v2:172.21.15.192:3300/0 conn(0x7f169c0d2030 0x7f169c0b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f16a3210640 1 --2- 172.21.15.163:0/4161724585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f169c1629e0 0x7f169c0d1af0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f16a3210640 1 -- 172.21.15.163:0/4161724585 >> 172.21.15.163:0/4161724585 conn(0x7f169c09e410 msgr2=0x7f169c0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f16a3210640 1 -- 172.21.15.163:0/4161724585 shutdown_connections 2024-05-08T06:36:51.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f78dbb0a640 1 -- 172.21.15.163:0/1847367010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78cc048b00 msgr2=0x7f78cc0526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f78dbb0a640 1 --2- 172.21.15.163:0/1847367010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78cc048b00 0x7f78cc0526b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009750 tx=0x7f78c402a010 comp rx=0 tx=0).stop 2024-05-08T06:36:51.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f78dbb0a640 1 -- 172.21.15.163:0/1847367010 shutdown_connections 2024-05-08T06:36:51.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f78dbb0a640 1 --2- 172.21.15.163:0/1847367010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78cc048b00 0x7f78cc0526b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f78dbb0a640 1 -- 172.21.15.163:0/1847367010 >> 172.21.15.163:0/1847367010 conn(0x7f78cc01e6f0 msgr2=0x7f78cc055420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/102044214 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ac15f4c0 msgr2=0x7fb5ac163020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7fb5b1c1d640 1 --2- 172.21.15.163:0/102044214 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ac15f4c0 0x7fb5ac163020 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009870 tx=0x7fb59c02a1c0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f16a3210640 1 -- 172.21.15.163:0/4161724585 wait complete. 2024-05-08T06:36:51.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f83ebfff640 1 -- 172.21.15.163:0/255631172 wait complete. 2024-05-08T06:36:51.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/102044214 shutdown_connections 2024-05-08T06:36:51.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7fb5b1c1d640 1 --2- 172.21.15.163:0/102044214 >> v2:172.21.15.192:3300/0 conn(0x7fb5ac0d0720 0x7fb5ac0d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7fb5b1c1d640 1 --2- 172.21.15.163:0/102044214 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ac15f4c0 0x7fb5ac163020 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/102044214 >> 172.21.15.163:0/102044214 conn(0x7fb5ac06d300 msgr2=0x7fb5ac06d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f5aa37fe640 1 -- 172.21.15.163:0/2641489395 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9002e070 con 0x7f5a940052e0 2024-05-08T06:36:51.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f5aa37fe640 1 -- 172.21.15.163:0/2641489395 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90005d30 con 0x7f5a940052e0 2024-05-08T06:36:51.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f5aa98c2640 1 -- 172.21.15.163:0/2641489395 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a940052e0 msgr2=0x7f5a9400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f5aa98c2640 1 --2- 172.21.15.163:0/2641489395 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a940052e0 0x7f5a9400b9c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000c240 tx=0x7f5a94005830 comp rx=0 tx=0).stop 2024-05-08T06:36:51.628 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7fe1553c1640 1 -- 172.21.15.163:0/1109463883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1500bab60 msgr2=0x7fe1500d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.628 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7fe1553c1640 1 --2- 172.21.15.163:0/1109463883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1500bab60 0x7fe1500d0010 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe14800b1a0 tx=0x7fe14802a8e0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.628 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f9706ffd640 1 -- 172.21.15.163:0/2031864973 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700007740 con 0x7f9708073cb0 2024-05-08T06:36:51.628 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f9706ffd640 1 -- 172.21.15.163:0/2031864973 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97000078e0 con 0x7f9708073cb0 2024-05-08T06:36:51.628 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f9706ffd640 1 -- 172.21.15.163:0/2031864973 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700007cb0 con 0x7f9708073cb0 2024-05-08T06:36:51.628 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f5aa98c2640 1 -- 172.21.15.163:0/2641489395 shutdown_connections 2024-05-08T06:36:51.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f5aa98c2640 1 --2- 172.21.15.163:0/2641489395 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a940052e0 0x7f5a9400b9c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f5aa98c2640 1 -- 172.21.15.163:0/2641489395 >> 172.21.15.163:0/2641489395 conn(0x7f5a9c055c50 msgr2=0x7f5a9c056060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f81b4c76640 1 --2- 172.21.15.163:0/3471879457 >> v2:172.21.15.163:3300/0 conn(0x7f81b00b4340 0x7f81b00b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f81b4c76640 1 -- 172.21.15.163:0/3471879457 >> v1:172.21.15.192:6789/0 conn(0x7f81b0072510 legacy=0x7f81b0072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f78dbb0a640 1 -- 172.21.15.163:0/1847367010 shutdown_connections 2024-05-08T06:36:51.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/102044214 shutdown_connections 2024-05-08T06:36:51.630 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f78dbb0a640 1 -- 172.21.15.163:0/1847367010 wait complete. 2024-05-08T06:36:51.630 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f5aa98c2640 1 -- 172.21.15.163:0/2641489395 shutdown_connections 2024-05-08T06:36:51.630 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f5aa98c2640 1 -- 172.21.15.163:0/2641489395 wait complete. 2024-05-08T06:36:51.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f970e41d640 1 -- 172.21.15.163:0/2031864973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9708073cb0 msgr2=0x7f9708074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f970e41d640 1 --2- 172.21.15.163:0/2031864973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9708073cb0 0x7f9708074120 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9700009d90 tx=0x7f970002a660 comp rx=0 tx=0).stop 2024-05-08T06:36:51.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f970e41d640 1 -- 172.21.15.163:0/2031864973 shutdown_connections 2024-05-08T06:36:51.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f970e41d640 1 --2- 172.21.15.163:0/2031864973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9708073cb0 0x7f9708074120 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f970e41d640 1 --2- 172.21.15.163:0/2031864973 >> v2:172.21.15.163:3300/0 conn(0x7f97080b4630 0x7f97080cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f970e41d640 1 -- 172.21.15.163:0/2031864973 >> 172.21.15.163:0/2031864973 conn(0x7f97080b29b0 msgr2=0x7f97080b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.593+0000 7f81b4c76640 1 -- 172.21.15.163:0/3471879457 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b006e830 con 0x7f81b00d1270 2024-05-08T06:36:51.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f81b5477640 1 --2- 172.21.15.163:0/3471879457 >> v2:172.21.15.163:3300/0 conn(0x7f81b00b4340 0x7f81b00b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:51.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7f81b4c76640 1 --2- 172.21.15.163:0/3471879457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81b00d1270 0x7f81b00d1650 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f81b00745a0 tx=0x7f81a802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=604aa02b0589f872 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.594+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/102044214 wait complete. 2024-05-08T06:36:51.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.595+0000 7f970e41d640 1 -- 172.21.15.163:0/2031864973 shutdown_connections 2024-05-08T06:36:51.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.595+0000 7f970e41d640 1 -- 172.21.15.163:0/2031864973 wait complete. 2024-05-08T06:36:51.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.595+0000 7f16a3210640 1 Processor -- start 2024-05-08T06:36:51.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.595+0000 7f16a3210640 1 -- start start 2024-05-08T06:36:51.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.595+0000 7f81af7fe640 1 -- 172.21.15.163:0/3471879457 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8007740 con 0x7f81b00d1270 2024-05-08T06:36:51.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.595+0000 7f81af7fe640 1 -- 172.21.15.163:0/3471879457 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a80078e0 con 0x7f81b00d1270 2024-05-08T06:36:51.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.595+0000 7f81af7fe640 1 -- 172.21.15.163:0/3471879457 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8007740 con 0x7f81b00d1270 2024-05-08T06:36:51.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.595+0000 7f83ebfff640 1 Processor -- start 2024-05-08T06:36:51.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7f5aa98c2640 1 Processor -- start 2024-05-08T06:36:51.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7f5aa98c2640 1 -- start start 2024-05-08T06:36:51.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.595+0000 7f83ebfff640 1 -- start start 2024-05-08T06:36:51.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7f78dbb0a640 1 Processor -- start 2024-05-08T06:36:51.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7f81b6eda640 1 -- 172.21.15.163:0/3471879457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81b00d1270 msgr2=0x7f81b00d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7f81b6eda640 1 --2- 172.21.15.163:0/3471879457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81b00d1270 0x7f81b00d1650 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f81b00745a0 tx=0x7f81a802a660 comp rx=0 tx=0).stop 2024-05-08T06:36:51.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7f78dbb0a640 1 -- start start 2024-05-08T06:36:51.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7fe1553c1640 1 -- 172.21.15.163:0/1109463883 shutdown_connections 2024-05-08T06:36:51.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7fe1553c1640 1 --2- 172.21.15.163:0/1109463883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1500bab60 0x7fe1500d0010 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7fe1553c1640 1 -- 172.21.15.163:0/1109463883 >> 172.21.15.163:0/1109463883 conn(0x7fe15006d300 msgr2=0x7fe15006d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7f81b6eda640 1 -- 172.21.15.163:0/3471879457 shutdown_connections 2024-05-08T06:36:51.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7f81b6eda640 1 --2- 172.21.15.163:0/3471879457 >> v2:172.21.15.163:3300/0 conn(0x7f81b00b4340 0x7f81b00b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7f81b6eda640 1 --2- 172.21.15.163:0/3471879457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81b00d1270 0x7f81b00d1650 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.637 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7f81b6eda640 1 -- 172.21.15.163:0/3471879457 >> 172.21.15.163:0/3471879457 conn(0x7f81b006c500 msgr2=0x7f81b006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.637 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7f81b6eda640 1 -- 172.21.15.163:0/3471879457 shutdown_connections 2024-05-08T06:36:51.637 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7f81b6eda640 1 -- 172.21.15.163:0/3471879457 wait complete. 2024-05-08T06:36:51.637 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7fb5b1c1d640 1 Processor -- start 2024-05-08T06:36:51.637 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7fb5b1c1d640 1 -- start start 2024-05-08T06:36:51.637 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7f16a3210640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f169c0b55d0 0x7f169c0cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.596+0000 7f970e41d640 1 Processor -- start 2024-05-08T06:36:51.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f16a3210640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f169c0d2030 0x7f169c0b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f16a3210640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c1629e0 0x7f169c1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f16a3210640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f169c0d4190 con 0x7f169c1629e0 2024-05-08T06:36:51.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f16a3210640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f169c0b4b70 con 0x7f169c0d2030 2024-05-08T06:36:51.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f16a3210640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f169c162dc0 con 0x7f169c0b55d0 2024-05-08T06:36:51.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f970e41d640 1 -- start start 2024-05-08T06:36:51.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f83ebfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83e4008840 0x7f83e40da5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f83ebfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83e4008c20 0x7f83e404f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f83ebfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83e40e40e0 0x7f83e4051ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f16a2a0f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c1629e0 0x7f169c1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.640 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f83ebfff640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f83e40e52b0 con 0x7f83e4008c20 2024-05-08T06:36:51.640 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f83ebfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f83e4057910 con 0x7f83e4008840 2024-05-08T06:36:51.640 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f83ebfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f83e40e5430 con 0x7f83e40e40e0 2024-05-08T06:36:51.640 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f16a2a0f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c1629e0 0x7f169c1542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44436/0 (socket says 172.21.15.163:44436) 2024-05-08T06:36:51.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f16a2a0f640 1 -- 172.21.15.163:0/625746159 learned_addr learned my addr 172.21.15.163:0/625746159 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f5aa98c2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a940052e0 0x7f5a9c1784c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f5aa98c2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a9c0e4ae0 0x7f5a9c0e4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f83ea7fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83e4008c20 0x7f83e404f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.641 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f83ea7fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83e4008c20 0x7f83e404f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44452/0 (socket says 172.21.15.163:44452) 2024-05-08T06:36:51.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f83ea7fc640 1 -- 172.21.15.163:0/2704293457 learned_addr learned my addr 172.21.15.163:0/2704293457 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f5aa98c2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a9c031bd0 0x7f5a9c032050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f5aa88c0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a940052e0 0x7f5a9c1784c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f5aa98c2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c0e36a0 con 0x7f5a940052e0 2024-05-08T06:36:51.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f5aa98c2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c054b80 con 0x7f5a9c031bd0 2024-05-08T06:36:51.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f5aa98c2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5a9c053820 con 0x7f5a9c0e4ae0 2024-05-08T06:36:51.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f5aa88c0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a940052e0 0x7f5a9c1784c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44468/0 (socket says 172.21.15.163:44468) 2024-05-08T06:36:51.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f5aa88c0640 1 -- 172.21.15.163:0/2302273947 learned_addr learned my addr 172.21.15.163:0/2302273947 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f78dbb0a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78cc03b0e0 0x7f78cc0dec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7fb5b1c1d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5ac0bab60 0x7fb5ac0b99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7fb5b1c1d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5ac0d0720 0x7fb5ac205250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7fb5b1c1d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ac15f4c0 0x7fb5ac205790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7fb5b1c1d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ac0d4210 con 0x7fb5ac0d0720 2024-05-08T06:36:51.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7fb5b1c1d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ac15f8a0 con 0x7fb5ac0bab60 2024-05-08T06:36:51.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7fb5b1c1d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ac0d4390 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f83eaffd640 1 --2- 172.21.15.163:0/2704293457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83e4008840 0x7f83e40da5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f78dbb0a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78cc0eaca0 0x7f78cc04df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f16a2a0f640 1 -- 172.21.15.163:0/625746159 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f169c0b55d0 msgr2=0x7f169c0cdd40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T06:36:51.645 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f78dbb0a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78cc1856e0 0x7f78cc0e0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.645 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7fb5abfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ac15f4c0 0x7fb5ac205790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.645 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f16a2a0f640 1 --2- 172.21.15.163:0/625746159 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f169c0b55d0 0x7f169c0cdd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.645 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f16a2a0f640 1 -- 172.21.15.163:0/625746159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f169c0d2030 msgr2=0x7f169c0b7d00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T06:36:51.646 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f16a2a0f640 1 --2- 172.21.15.163:0/625746159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f169c0d2030 0x7f169c0b7d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.646 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7fb5abfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ac15f4c0 0x7fb5ac205790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:48206/0 (socket says 172.21.15.163:48206) 2024-05-08T06:36:51.646 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7fb5abfff640 1 -- 172.21.15.163:0/2559308876 learned_addr learned my addr 172.21.15.163:0/2559308876 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.646 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f78dbb0a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc03a0d0 con 0x7f78cc03b0e0 2024-05-08T06:36:51.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f16a2a0f640 1 -- 172.21.15.163:0/625746159 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169c0b87c0 con 0x7f169c1629e0 2024-05-08T06:36:51.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f78dbb0a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc0eb290 con 0x7f78cc1856e0 2024-05-08T06:36:51.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f78dbb0a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f78cc03ac50 con 0x7f78cc0eaca0 2024-05-08T06:36:51.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f78da0a7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78cc1856e0 0x7f78cc0e0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f78da0a7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78cc1856e0 0x7f78cc0e0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35650/0 (socket says 172.21.15.163:35650) 2024-05-08T06:36:51.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f78da0a7640 1 -- 172.21.15.163:0/627740669 learned_addr learned my addr 172.21.15.163:0/627740669 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.597+0000 7f83eb7fe640 1 --2- 172.21.15.163:0/2704293457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83e40e40e0 0x7f83e4051ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f83ea7fc640 1 -- 172.21.15.163:0/2704293457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83e40e40e0 msgr2=0x7f83e4051ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f83ea7fc640 1 --2- 172.21.15.163:0/2704293457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83e40e40e0 0x7f83e4051ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f83ea7fc640 1 -- 172.21.15.163:0/2704293457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83e4008840 msgr2=0x7f83e40da5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f83ea7fc640 1 --2- 172.21.15.163:0/2704293457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83e4008840 0x7f83e40da5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f16a2a0f640 1 --2- 172.21.15.163:0/625746159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c1629e0 0x7f169c1542e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f168c00b410 tx=0x7f168c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f83ea7fc640 1 -- 172.21.15.163:0/2704293457 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e404ff50 con 0x7f83e4008c20 2024-05-08T06:36:51.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7fb5ab7fe640 1 --2- 172.21.15.163:0/2559308876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5ac0bab60 0x7fb5ac0b99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7fb5abfff640 1 -- 172.21.15.163:0/2559308876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5ac0bab60 msgr2=0x7fb5ac0b99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7fb5abfff640 1 --2- 172.21.15.163:0/2559308876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5ac0bab60 0x7fb5ac0b99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7fb5abfff640 1 -- 172.21.15.163:0/2559308876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5ac0d0720 msgr2=0x7fb5ac205250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7fb5abfff640 1 --2- 172.21.15.163:0/2559308876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5ac0d0720 0x7fb5ac205250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7fb5abfff640 1 -- 172.21.15.163:0/2559308876 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ac206180 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7fb5ab7fe640 1 --2- 172.21.15.163:0/2559308876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5ac0bab60 0x7fb5ac0b99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:51.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f78d90a5640 1 --2- 172.21.15.163:0/627740669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78cc0eaca0 0x7f78cc04df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f78da0a7640 1 -- 172.21.15.163:0/627740669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78cc0eaca0 msgr2=0x7f78cc04df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f78da0a7640 1 --2- 172.21.15.163:0/627740669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78cc0eaca0 0x7f78cc04df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f78da0a7640 1 -- 172.21.15.163:0/627740669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78cc03b0e0 msgr2=0x7f78cc0dec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f78da0a7640 1 --2- 172.21.15.163:0/627740669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78cc03b0e0 0x7f78cc0dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f78da0a7640 1 -- 172.21.15.163:0/627740669 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc04e6c0 con 0x7f78cc1856e0 2024-05-08T06:36:51.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f81b6eda640 1 Processor -- start 2024-05-08T06:36:51.652 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7fb5abfff640 1 --2- 172.21.15.163:0/2559308876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ac15f4c0 0x7fb5ac205790 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400efc0 tx=0x7fb5a400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.652 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f1692ffd640 1 -- 172.21.15.163:0/625746159 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c015030 con 0x7f169c1629e0 2024-05-08T06:36:51.652 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f5aa3fff640 1 --2- 172.21.15.163:0/2302273947 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a9c0e4ae0 0x7f5a9c0e4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f81b6eda640 1 -- start start 2024-05-08T06:36:51.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f5aa90c1640 1 --2- 172.21.15.163:0/2302273947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a9c031bd0 0x7f5a9c032050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f1692ffd640 1 -- 172.21.15.163:0/625746159 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f168c004020 con 0x7f169c1629e0 2024-05-08T06:36:51.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f16a3210640 1 -- 172.21.15.163:0/625746159 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169c0b8aa0 con 0x7f169c1629e0 2024-05-08T06:36:51.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f5aa88c0640 1 -- 172.21.15.163:0/2302273947 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a9c0e4ae0 msgr2=0x7f5a9c0e4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f5aa88c0640 1 --2- 172.21.15.163:0/2302273947 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a9c0e4ae0 0x7f5a9c0e4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f5aa88c0640 1 -- 172.21.15.163:0/2302273947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a9c031bd0 msgr2=0x7f5a9c032050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f5aa88c0640 1 --2- 172.21.15.163:0/2302273947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a9c031bd0 0x7f5a9c032050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f5aa88c0640 1 -- 172.21.15.163:0/2302273947 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9c032970 con 0x7f5a940052e0 2024-05-08T06:36:51.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f5aa88c0640 1 --2- 172.21.15.163:0/2302273947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a940052e0 0x7f5a9c1784c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5a980108f0 tx=0x7f5a98010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f81b6eda640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81b0072510 0x7f81b0154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f81b6eda640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81b00b4340 0x7f81b0161970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f81b6eda640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81b00d1270 0x7f81b0161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f81b6eda640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f81b006f0d0 con 0x7f81b0072510 2024-05-08T06:36:51.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f81b6eda640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f81b006edf0 con 0x7f81b00b4340 2024-05-08T06:36:51.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f81b6eda640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f81b0166fb0 con 0x7f81b00d1270 2024-05-08T06:36:51.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f78da0a7640 1 --2- 172.21.15.163:0/627740669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78cc1856e0 0x7f78cc0e0560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f78d000c500 tx=0x7f78d000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f78ca7fc640 1 -- 172.21.15.163:0/627740669 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0017070 con 0x7f78cc1856e0 2024-05-08T06:36:51.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f78ca7fc640 1 -- 172.21.15.163:0/627740669 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d00035f0 con 0x7f78cc1856e0 2024-05-08T06:36:51.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f78ca7fc640 1 -- 172.21.15.163:0/627740669 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d00039c0 con 0x7f78cc1856e0 2024-05-08T06:36:51.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7fb5a3fff640 1 -- 172.21.15.163:0/2559308876 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4018020 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7fb5a3fff640 1 -- 172.21.15.163:0/2559308876 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a4014480 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7fb5a3fff640 1 -- 172.21.15.163:0/2559308876 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4014870 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f1692ffd640 1 -- 172.21.15.163:0/625746159 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c004410 con 0x7f169c1629e0 2024-05-08T06:36:51.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f16a3210640 1 -- 172.21.15.163:0/625746159 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169c0b9060 con 0x7f169c1629e0 2024-05-08T06:36:51.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7fe1553c1640 1 -- 172.21.15.163:0/1109463883 shutdown_connections 2024-05-08T06:36:51.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7fe1553c1640 1 -- 172.21.15.163:0/1109463883 wait complete. 2024-05-08T06:36:51.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f83ea7fc640 1 --2- 172.21.15.163:0/2704293457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83e4008c20 0x7f83e404f720 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f83e000b480 tx=0x7f83e000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/2559308876 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ac205ea0 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f83d3fff640 1 -- 172.21.15.163:0/2704293457 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0014070 con 0x7f83e4008c20 2024-05-08T06:36:51.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f83d3fff640 1 -- 172.21.15.163:0/2704293457 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e0010890 con 0x7f83e4008c20 2024-05-08T06:36:51.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f83d3fff640 1 -- 172.21.15.163:0/2704293457 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0010c80 con 0x7f83e4008c20 2024-05-08T06:36:51.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f81b5477640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81b00d1270 0x7f81b0161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f81affff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81b00b4340 0x7f81b0161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f83ebfff640 1 -- 172.21.15.163:0/2704293457 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e40d9b10 con 0x7f83e4008c20 2024-05-08T06:36:51.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/2559308876 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ac2020e0 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f5aa17fa640 1 -- 172.21.15.163:0/2302273947 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9801a020 con 0x7f5a940052e0 2024-05-08T06:36:51.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f81affff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81b00b4340 0x7f81b0161970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35678/0 (socket says 172.21.15.163:35678) 2024-05-08T06:36:51.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f81affff640 1 -- 172.21.15.163:0/3192259881 learned_addr learned my addr 172.21.15.163:0/3192259881 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f5aa98c2640 1 -- 172.21.15.163:0/2302273947 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9c032c50 con 0x7f5a940052e0 2024-05-08T06:36:51.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f83ebfff640 1 -- 172.21.15.163:0/2704293457 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e404fc60 con 0x7f83e4008c20 2024-05-08T06:36:51.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/2559308876 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb5ac06a550 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f5aa98c2640 1 -- 172.21.15.163:0/2302273947 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a9c033210 con 0x7f5a940052e0 2024-05-08T06:36:51.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.601+0000 7f78dbb0a640 1 -- 172.21.15.163:0/627740669 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78cc04e9a0 con 0x7f78cc1856e0 2024-05-08T06:36:51.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.601+0000 7f78dbb0a640 1 -- 172.21.15.163:0/627740669 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78cc185b90 con 0x7f78cc1856e0 2024-05-08T06:36:51.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.601+0000 7fe1553c1640 1 Processor -- start 2024-05-08T06:36:51.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.601+0000 7f81b4c76640 1 --2- 172.21.15.163:0/3192259881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81b0072510 0x7f81b0154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.602+0000 7f81b5477640 1 -- 172.21.15.163:0/3192259881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81b00b4340 msgr2=0x7f81b0161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.602+0000 7f81b5477640 1 --2- 172.21.15.163:0/3192259881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81b00b4340 0x7f81b0161970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.602+0000 7f81b5477640 1 -- 172.21.15.163:0/3192259881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81b0072510 msgr2=0x7f81b0154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.602+0000 7f81b5477640 1 --2- 172.21.15.163:0/3192259881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81b0072510 0x7f81b0154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.602+0000 7f81b5477640 1 -- 172.21.15.163:0/3192259881 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b01fe060 con 0x7f81b00d1270 2024-05-08T06:36:51.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.602+0000 7f81b4c76640 1 --2- 172.21.15.163:0/3192259881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81b0072510 0x7f81b0154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:51.665 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.602+0000 7f16a3210640 1 -- 172.21.15.163:0/625746159 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f169c06a3d0 con 0x7f169c1629e0 2024-05-08T06:36:51.665 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.601+0000 7fe1553c1640 1 -- start start 2024-05-08T06:36:51.665 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.602+0000 7f83d1ffb640 1 -- 172.21.15.163:0/2704293457 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83e40057e0 con 0x7f83e4008c20 2024-05-08T06:36:51.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.602+0000 7f81b5477640 1 --2- 172.21.15.163:0/3192259881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81b00d1270 0x7f81b0161eb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f81a000e600 tx=0x7f81a000eac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.602+0000 7f1692ffd640 1 -- 172.21.15.163:0/625746159 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f168c018070 con 0x7f169c1629e0 2024-05-08T06:36:51.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.601+0000 7f5aa17fa640 1 -- 172.21.15.163:0/2302273947 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9800e9c0 con 0x7f5a940052e0 2024-05-08T06:36:51.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.602+0000 7f5a8effd640 1 -- 172.21.15.163:0/2302273947 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5a84001030 con 0x7f5a940052e0 2024-05-08T06:36:51.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7f5aa17fa640 1 -- 172.21.15.163:0/2302273947 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9801a020 con 0x7f5a940052e0 2024-05-08T06:36:51.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7f1692ffd640 1 --2- 172.21.15.163:0/625746159 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f16800762d0 0x7f1680078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f970e41d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9708073cb0 0x7f970813f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f970e41d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97080b4630 0x7f97080af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f970e41d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97080cdbd0 0x7f97080afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.598+0000 7f9707fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9708073cb0 0x7f970813f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f9707fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9708073cb0 0x7f970813f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35672/0 (socket says 172.21.15.163:35672) 2024-05-08T06:36:51.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f9707fff640 1 -- 172.21.15.163:0/3774413612 learned_addr learned my addr 172.21.15.163:0/3774413612 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f97077fe640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97080b4630 0x7f97080af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f970c9ba640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97080cdbd0 0x7f97080afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f970e41d640 1 -- 172.21.15.163:0/3774413612 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f970806fb00 con 0x7f97080cdbd0 2024-05-08T06:36:51.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f970e41d640 1 -- 172.21.15.163:0/3774413612 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9708073790 con 0x7f9708073cb0 2024-05-08T06:36:51.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7fe1553c1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1500bab60 0x7fe1500b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.599+0000 7f970e41d640 1 -- 172.21.15.163:0/3774413612 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f970806fc80 con 0x7f97080b4630 2024-05-08T06:36:51.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f9707fff640 1 -- 172.21.15.163:0/3774413612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97080b4630 msgr2=0x7f97080af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f9707fff640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97080b4630 0x7f97080af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f9707fff640 1 -- 172.21.15.163:0/3774413612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97080cdbd0 msgr2=0x7f97080afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f9707fff640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97080cdbd0 0x7f97080afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7fe1553c1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1500d0550 0x7fe1500b7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f9707fff640 1 -- 172.21.15.163:0/3774413612 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97080b0190 con 0x7f9708073cb0 2024-05-08T06:36:51.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f970c9ba640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97080cdbd0 0x7f97080afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:51.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f9707fff640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9708073cb0 0x7f970813f160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00d5f0 tx=0x7f96fc00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f9704ff9640 1 -- 172.21.15.163:0/3774413612 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0125f0 con 0x7f9708073cb0 2024-05-08T06:36:51.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.600+0000 7f9704ff9640 1 -- 172.21.15.163:0/3774413612 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96fc019070 con 0x7f9708073cb0 2024-05-08T06:36:51.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.601+0000 7f9704ff9640 1 -- 172.21.15.163:0/3774413612 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0129b0 con 0x7f9708073cb0 2024-05-08T06:36:51.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.602+0000 7f970e41d640 1 -- 172.21.15.163:0/3774413612 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97080ab690 con 0x7f9708073cb0 2024-05-08T06:36:51.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.602+0000 7f970e41d640 1 -- 172.21.15.163:0/3774413612 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97080abc50 con 0x7f9708073cb0 2024-05-08T06:36:51.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7f96ee7fc640 1 -- 172.21.15.163:0/3774413612 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f970806a3d0 con 0x7f9708073cb0 2024-05-08T06:36:51.673 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7fe1553c1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe15015f4c0 0x7fe1500cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7fe1553c1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe1500d4560 con 0x7fe1500bab60 2024-05-08T06:36:51.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7fe1553c1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1500d43e0 con 0x7fe1500d0550 2024-05-08T06:36:51.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7fe1553c1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe1500d46e0 con 0x7fe15015f4c0 2024-05-08T06:36:51.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7f78dbb0a640 1 -- 172.21.15.163:0/627740669 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78cc04eec0 con 0x7f78cc1856e0 2024-05-08T06:36:51.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7f81ad7fa640 1 -- 172.21.15.163:0/3192259881 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a000f040 con 0x7f81b00d1270 2024-05-08T06:36:51.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7f81ad7fa640 1 -- 172.21.15.163:0/3192259881 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a0004230 con 0x7f81b00d1270 2024-05-08T06:36:51.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7f81ad7fa640 1 -- 172.21.15.163:0/3192259881 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0003610 con 0x7f81b00d1270 2024-05-08T06:36:51.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7f81b6eda640 1 -- 172.21.15.163:0/3192259881 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b01fd2a0 con 0x7f81b00d1270 2024-05-08T06:36:51.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7f81b6eda640 1 -- 172.21.15.163:0/3192259881 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b01fd860 con 0x7f81b00d1270 2024-05-08T06:36:51.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7f78ca7fc640 1 -- 172.21.15.163:0/627740669 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f78d001a070 con 0x7f78cc1856e0 2024-05-08T06:36:51.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.603+0000 7f16a220e640 1 --2- 172.21.15.163:0/625746159 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f16800762d0 0x7f1680078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.678 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.604+0000 7f81b6eda640 1 -- 172.21.15.163:0/3192259881 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f81b006a550 con 0x7f81b00d1270 2024-05-08T06:36:51.678 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.604+0000 7f16a220e640 1 --2- 172.21.15.163:0/625746159 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f16800762d0 0x7f1680078790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1698034000 tx=0x7f169802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.604+0000 7fe14f7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe15015f4c0 0x7fe1500cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.604+0000 7fe14f7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe15015f4c0 0x7fe1500cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:48254/0 (socket says 172.21.15.163:48254) 2024-05-08T06:36:51.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.604+0000 7fe14f7fe640 1 -- 172.21.15.163:0/2067982890 learned_addr learned my addr 172.21.15.163:0/2067982890 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.604+0000 7fe14e7fc640 1 --2- 172.21.15.163:0/2067982890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1500d0550 0x7fe1500b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.604+0000 7fe14effd640 1 --2- 172.21.15.163:0/2067982890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1500bab60 0x7fe1500b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.604+0000 7f78ca7fc640 1 --2- 172.21.15.163:0/627740669 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f78b80763a0 0x7f78b8078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.604+0000 7f78ca7fc640 1 -- 172.21.15.163:0/627740669 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f78d0096a00 con 0x7f78cc1856e0 2024-05-08T06:36:51.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.604+0000 7f78ca7fc640 1 --2- 172.21.15.163:0/627740669 >> [v2:172.21.15.163:6808/601639604,v1:172.21.15.163:6809/601639604] conn(0x7f78b807e5a0 0x7f78b80809c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.604+0000 7f78ca7fc640 1 -- 172.21.15.163:0/627740669 --> [v2:172.21.15.163:6808/601639604,v1:172.21.15.163:6809/601639604] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78b8081070 con 0x7f78b807e5a0 2024-05-08T06:36:51.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7f78ca7fc640 1 -- 172.21.15.163:0/627740669 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f78d005c320 con 0x7f78cc1856e0 2024-05-08T06:36:51.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7f78d90a5640 1 --2- 172.21.15.163:0/627740669 >> [v2:172.21.15.163:6808/601639604,v1:172.21.15.163:6809/601639604] conn(0x7f78b807e5a0 0x7f78b80809c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.604+0000 7fb5a3fff640 1 -- 172.21.15.163:0/2559308876 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb5a401b020 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7f78d98a6640 1 --2- 172.21.15.163:0/627740669 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f78b80763a0 0x7f78b8078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7fb5a3fff640 1 --2- 172.21.15.163:0/2559308876 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb58c0763d0 0x7fb58c078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7fe14effd640 1 -- 172.21.15.163:0/2067982890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe15015f4c0 msgr2=0x7fe1500cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7fe14effd640 1 --2- 172.21.15.163:0/2067982890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe15015f4c0 0x7fe1500cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7fe14effd640 1 -- 172.21.15.163:0/2067982890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1500d0550 msgr2=0x7fe1500b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7fe14effd640 1 --2- 172.21.15.163:0/2067982890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1500d0550 0x7fe1500b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7fe14effd640 1 -- 172.21.15.163:0/2067982890 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1500cc850 con 0x7fe1500bab60 2024-05-08T06:36:51.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7fb5a3fff640 1 -- 172.21.15.163:0/2559308876 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb5a4098620 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7fb5ab7fe640 1 --2- 172.21.15.163:0/2559308876 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb58c0763d0 0x7fb58c078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7fe14effd640 1 --2- 172.21.15.163:0/2067982890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1500bab60 0x7fe1500b5930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe14000b410 tx=0x7fe14000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7f83d3fff640 1 -- 172.21.15.163:0/2704293457 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f83e0017070 con 0x7f83e4008c20 2024-05-08T06:36:51.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7f1692ffd640 1 -- 172.21.15.163:0/625746159 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f168c0957a0 con 0x7f169c1629e0 2024-05-08T06:36:51.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.604+0000 7f9704ff9640 1 -- 172.21.15.163:0/3774413612 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f96fc012b50 con 0x7f9708073cb0 2024-05-08T06:36:51.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7f9704ff9640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f96f40763a0 0x7f96f4078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7f9704ff9640 1 -- 172.21.15.163:0/3774413612 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f96fc0986c0 con 0x7f9708073cb0 2024-05-08T06:36:51.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7f9704ff9640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x7f96f407e5c0 0x7f96f40809e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7f9704ff9640 1 -- 172.21.15.163:0/3774413612 --> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96fc001d30 con 0x7f96f407e5c0 2024-05-08T06:36:51.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.605+0000 7f9704ff9640 1 -- 172.21.15.163:0/3774413612 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f96fc098ab0 con 0x7f9708073cb0 2024-05-08T06:36:51.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7f1692ffd640 1 --2- 172.21.15.163:0/625746159 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x7f168007e470 0x7f1680080890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7f1692ffd640 1 -- 172.21.15.163:0/625746159 --> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f168c00fe50 con 0x7f168007e470 2024-05-08T06:36:51.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7fb5a3fff640 1 --2- 172.21.15.163:0/2559308876 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x7fb58c07e5f0 0x7fb58c080a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7fb5a3fff640 1 -- 172.21.15.163:0/2559308876 --> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb58c0810c0 con 0x7fb58c07e5f0 2024-05-08T06:36:51.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7fb5a3fff640 1 -- 172.21.15.163:0/2559308876 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb5a4098a10 con 0x7fb5ac15f4c0 2024-05-08T06:36:51.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7f1692ffd640 1 -- 172.21.15.163:0/625746159 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f168c05ca10 con 0x7f169c1629e0 2024-05-08T06:36:51.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7fe1377fe640 1 -- 172.21.15.163:0/2067982890 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140015030 con 0x7fe1500bab60 2024-05-08T06:36:51.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7f78d90a5640 1 --2- 172.21.15.163:0/627740669 >> [v2:172.21.15.163:6808/601639604,v1:172.21.15.163:6809/601639604] conn(0x7f78b807e5a0 0x7f78b80809c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7f78d98a6640 1 --2- 172.21.15.163:0/627740669 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f78b80763a0 0x7f78b8078860 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78d4065ed0 tx=0x7f78d407bc20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7fe1553c1640 1 -- 172.21.15.163:0/2067982890 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1500b5fc0 con 0x7fe1500bab60 2024-05-08T06:36:51.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7fb5aaffd640 1 --2- 172.21.15.163:0/2559308876 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x7fb58c07e5f0 0x7fb58c080a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7fe1553c1640 1 -- 172.21.15.163:0/2067982890 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1500b6580 con 0x7fe1500bab60 2024-05-08T06:36:51.693 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7fe1377fe640 1 -- 172.21.15.163:0/2067982890 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe140004020 con 0x7fe1500bab60 2024-05-08T06:36:51.693 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7fb5ab7fe640 1 --2- 172.21.15.163:0/2559308876 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb58c0763d0 0x7fb58c078890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb59c034000 tx=0x7fb59c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.693 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7fb5aaffd640 1 --2- 172.21.15.163:0/2559308876 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x7fb58c07e5f0 0x7fb58c080a10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.607+0000 7f78ca7fc640 1 -- 172.21.15.163:0/627740669 <== osd.2 v2:172.21.15.163:6808/601639604 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78b8081070 con 0x7f78b807e5a0 2024-05-08T06:36:51.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.607+0000 7fe1377fe640 1 -- 172.21.15.163:0/2067982890 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140004410 con 0x7fe1500bab60 2024-05-08T06:36:51.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.607+0000 7fe1553c1640 1 -- 172.21.15.163:0/2067982890 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe130001030 con 0x7fe1500bab60 2024-05-08T06:36:51.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.607+0000 7fb5a3fff640 1 -- 172.21.15.163:0/2559308876 <== osd.3 v2:172.21.15.163:6816/127592521 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb58c0810c0 con 0x7fb58c07e5f0 2024-05-08T06:36:51.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7f16a1a0d640 1 --2- 172.21.15.163:0/625746159 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x7f168007e470 0x7f1680080890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7f16a1a0d640 1 --2- 172.21.15.163:0/625746159 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x7f168007e470 0x7f1680080890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.607+0000 7f1692ffd640 1 -- 172.21.15.163:0/625746159 <== osd.4 v2:172.21.15.192:6824/272496966 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f168c00fe50 con 0x7f168007e470 2024-05-08T06:36:51.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7f5aa17fa640 1 -- 172.21.15.163:0/2302273947 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5a9801d020 con 0x7f5a940052e0 2024-05-08T06:36:51.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.607+0000 7f5aa17fa640 1 --2- 172.21.15.163:0/2302273947 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5a88076700 0x7f5a88078bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.607+0000 7f5aa17fa640 1 -- 172.21.15.163:0/2302273947 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5a9800f040 con 0x7f5a940052e0 2024-05-08T06:36:51.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.608+0000 7f5aa17fa640 1 --2- 172.21.15.163:0/2302273947 >> [v2:172.21.15.163:6800/3059241761,v1:172.21.15.163:6801/3059241761] conn(0x7f5a8807e8a0 0x7f5a88080cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.608+0000 7f5aa17fa640 1 -- 172.21.15.163:0/2302273947 --> [v2:172.21.15.163:6800/3059241761,v1:172.21.15.163:6801/3059241761] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a9800af30 con 0x7f5a8807e8a0 2024-05-08T06:36:51.697 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.608+0000 7f5aa17fa640 1 -- 172.21.15.163:0/2302273947 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5a98098070 con 0x7f5a940052e0 2024-05-08T06:36:51.697 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7f97077fe640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f96f40763a0 0x7f96f4078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.697 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.606+0000 7f970c9ba640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x7f96f407e5c0 0x7f96f40809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.697 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.608+0000 7f5aa3fff640 1 --2- 172.21.15.163:0/2302273947 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5a88076700 0x7f5a88078bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.697 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.608+0000 7f83d3fff640 1 --2- 172.21.15.163:0/2704293457 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f83d80763a0 0x7f83d8078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.697 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.608+0000 7f83d3fff640 1 -- 172.21.15.163:0/2704293457 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f83e0097340 con 0x7f83e4008c20 2024-05-08T06:36:51.698 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.608+0000 7f83d3fff640 1 --2- 172.21.15.163:0/2704293457 >> [v2:172.21.15.192:6808/1747004047,v1:172.21.15.192:6809/1747004047] conn(0x7f83d807e620 0x7f83d8080a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.698 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.608+0000 7f83d3fff640 1 -- 172.21.15.163:0/2704293457 --> [v2:172.21.15.192:6808/1747004047,v1:172.21.15.192:6809/1747004047] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83e001ae50 con 0x7f83d807e620 2024-05-08T06:36:51.698 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.608+0000 7f83d3fff640 1 -- 172.21.15.163:0/2704293457 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f83e009e050 con 0x7f83e4008c20 2024-05-08T06:36:51.698 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.608+0000 7f81ad7fa640 1 -- 172.21.15.163:0/3192259881 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f81a00037b0 con 0x7f81b00d1270 2024-05-08T06:36:51.698 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.608+0000 7f97077fe640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f96f40763a0 0x7f96f4078860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96f8007f30 tx=0x7f96f8007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.698 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.608+0000 7f83eb7fe640 1 --2- 172.21.15.163:0/2704293457 >> [v2:172.21.15.192:6808/1747004047,v1:172.21.15.192:6809/1747004047] conn(0x7f83d807e620 0x7f83d8080a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.699 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.608+0000 7f5aa90c1640 1 --2- 172.21.15.163:0/2302273947 >> [v2:172.21.15.163:6800/3059241761,v1:172.21.15.163:6801/3059241761] conn(0x7f5a8807e8a0 0x7f5a88080cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.699 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.608+0000 7fe1377fe640 1 -- 172.21.15.163:0/2067982890 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe140018070 con 0x7fe1500bab60 2024-05-08T06:36:51.699 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.609+0000 7f83eaffd640 1 --2- 172.21.15.163:0/2704293457 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f83d80763a0 0x7f83d8078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.609+0000 7f5aa3fff640 1 --2- 172.21.15.163:0/2302273947 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5a88076700 0x7f5a88078bc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4065ed0 tx=0x7f5aa406ad80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.609+0000 7f5aa90c1640 1 --2- 172.21.15.163:0/2302273947 >> [v2:172.21.15.163:6800/3059241761,v1:172.21.15.163:6801/3059241761] conn(0x7f5a8807e8a0 0x7f5a88080cc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.609+0000 7f83eb7fe640 1 --2- 172.21.15.163:0/2704293457 >> [v2:172.21.15.192:6808/1747004047,v1:172.21.15.192:6809/1747004047] conn(0x7f83d807e620 0x7f83d8080a40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.609+0000 7f83eaffd640 1 --2- 172.21.15.163:0/2704293457 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f83d80763a0 0x7f83d8078860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f83dc000f60 tx=0x7f83dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.701 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.609+0000 7fe1377fe640 1 --2- 172.21.15.163:0/2067982890 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe13c0763a0 0x7fe13c078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.701 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.609+0000 7fe1377fe640 1 -- 172.21.15.163:0/2067982890 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe140097870 con 0x7fe1500bab60 2024-05-08T06:36:51.701 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7f81ad7fa640 1 --2- 172.21.15.163:0/3192259881 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f819c0763d0 0x7f819c078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.701 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7fe14e7fc640 1 --2- 172.21.15.163:0/2067982890 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe13c0763a0 0x7fe13c078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.701 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7fe1377fe640 1 --2- 172.21.15.163:0/2067982890 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x7fe13c07e5c0 0x7fe13c0809e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.701 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7fe1377fe640 1 -- 172.21.15.163:0/2067982890 --> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe13c081090 con 0x7fe13c07e5c0 2024-05-08T06:36:51.702 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7fe1377fe640 1 -- 172.21.15.163:0/2067982890 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe140097c60 con 0x7fe1500bab60 2024-05-08T06:36:51.702 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7f970c9ba640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x7f96f407e5c0 0x7f96f40809e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7f81ad7fa640 1 -- 172.21.15.163:0/3192259881 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f81a0097590 con 0x7f81b00d1270 2024-05-08T06:36:51.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7f81b4c76640 1 --2- 172.21.15.163:0/3192259881 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f819c0763d0 0x7f819c078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7f81ad7fa640 1 --2- 172.21.15.163:0/3192259881 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x7f819c07e5f0 0x7f819c080a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7f81ad7fa640 1 -- 172.21.15.163:0/3192259881 --> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81a0008eb0 con 0x7f819c07e5f0 2024-05-08T06:36:51.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7f81ad7fa640 1 -- 172.21.15.163:0/3192259881 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f81a0097980 con 0x7f81b00d1270 2024-05-08T06:36:51.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7f83d3fff640 1 -- 172.21.15.163:0/2704293457 <== osd.7 v2:172.21.15.192:6808/1747004047 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83e001ae50 con 0x7f83d807e620 2024-05-08T06:36:51.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7fe14f7fe640 1 --2- 172.21.15.163:0/2067982890 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x7fe13c07e5c0 0x7fe13c0809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7f5aa17fa640 1 -- 172.21.15.163:0/2302273947 <== osd.0 v2:172.21.15.163:6800/3059241761 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a9800af30 con 0x7f5a8807e8a0 2024-05-08T06:36:51.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7fe14e7fc640 1 --2- 172.21.15.163:0/2067982890 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe13c0763a0 0x7fe13c078860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe148034000 tx=0x7fe148009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.610+0000 7f81affff640 1 --2- 172.21.15.163:0/3192259881 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x7f819c07e5f0 0x7f819c080a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.611+0000 7f81b4c76640 1 --2- 172.21.15.163:0/3192259881 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f819c0763d0 0x7f819c078890 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f81a8034000 tx=0x7f81a8033040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.611+0000 7f9704ff9640 1 -- 172.21.15.163:0/3774413612 <== osd.6 v2:172.21.15.192:6816/1074108675 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96fc001d30 con 0x7f96f407e5c0 2024-05-08T06:36:51.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.611+0000 7f81affff640 1 --2- 172.21.15.163:0/3192259881 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x7f819c07e5f0 0x7f819c080a10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.612+0000 7fe14f7fe640 1 --2- 172.21.15.163:0/2067982890 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x7fe13c07e5c0 0x7fe13c0809e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.612+0000 7f81ad7fa640 1 -- 172.21.15.163:0/3192259881 <== osd.1 v2:172.21.15.163:6824/2322245857 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f81a0008eb0 con 0x7f819c07e5f0 2024-05-08T06:36:51.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.613+0000 7fe1377fe640 1 -- 172.21.15.163:0/2067982890 <== osd.5 v2:172.21.15.192:6800/2911825781 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe13c081090 con 0x7fe13c07e5c0 2024-05-08T06:36:51.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.638+0000 7f78dbb0a640 1 -- 172.21.15.163:0/627740669 --> [v2:172.21.15.163:6808/601639604,v1:172.21.15.163:6809/601639604] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f78cc050ab0 con 0x7f78b807e5a0 2024-05-08T06:36:51.708 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.638+0000 7f78ca7fc640 1 -- 172.21.15.163:0/627740669 <== osd.2 v2:172.21.15.163:6808/601639604 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f78cc050ab0 con 0x7f78b807e5a0 2024-05-08T06:36:51.708 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.639+0000 7f78b7fff640 1 -- 172.21.15.163:0/627740669 >> [v2:172.21.15.163:6808/601639604,v1:172.21.15.163:6809/601639604] conn(0x7f78b807e5a0 msgr2=0x7f78b80809c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.709 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.639+0000 7f78b7fff640 1 --2- 172.21.15.163:0/627740669 >> [v2:172.21.15.163:6808/601639604,v1:172.21.15.163:6809/601639604] conn(0x7f78b807e5a0 0x7f78b80809c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.709 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.639+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/2559308876 --> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb5ac0d2e70 con 0x7fb58c07e5f0 2024-05-08T06:36:51.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.640+0000 7f78b7fff640 1 -- 172.21.15.163:0/627740669 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f78b80763a0 msgr2=0x7f78b8078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.640+0000 7f78b7fff640 1 --2- 172.21.15.163:0/627740669 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f78b80763a0 0x7f78b8078860 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78d4065ed0 tx=0x7f78d407bc20 comp rx=0 tx=0).stop 2024-05-08T06:36:51.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.640+0000 7f78b7fff640 1 -- 172.21.15.163:0/627740669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78cc1856e0 msgr2=0x7f78cc0e0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.640+0000 7f78b7fff640 1 --2- 172.21.15.163:0/627740669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78cc1856e0 0x7f78cc0e0560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f78d000c500 tx=0x7f78d000c9c0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.640+0000 7f78da0a7640 1 -- 172.21.15.163:0/627740669 reap_dead start 2024-05-08T06:36:51.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.640+0000 7f78b7fff640 1 -- 172.21.15.163:0/627740669 shutdown_connections 2024-05-08T06:36:51.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.640+0000 7f78b7fff640 1 -- 172.21.15.163:0/627740669 >> 172.21.15.163:0/627740669 conn(0x7f78cc01e6f0 msgr2=0x7f78cc0e9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.640+0000 7f78b7fff640 1 -- 172.21.15.163:0/627740669 shutdown_connections 2024-05-08T06:36:51.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.640+0000 7f78b7fff640 1 -- 172.21.15.163:0/627740669 wait complete. 2024-05-08T06:36:51.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.640+0000 7fb5a3fff640 1 -- 172.21.15.163:0/2559308876 <== osd.3 v2:172.21.15.163:6816/127592521 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb5ac0d2e70 con 0x7fb58c07e5f0 2024-05-08T06:36:51.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.640+0000 7f16a3210640 1 -- 172.21.15.163:0/625746159 --> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f169c06ecb0 con 0x7f168007e470 2024-05-08T06:36:51.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.641+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/2559308876 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x7fb58c07e5f0 msgr2=0x7fb58c080a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.641+0000 7fb5b1c1d640 1 --2- 172.21.15.163:0/2559308876 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x7fb58c07e5f0 0x7fb58c080a10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.641+0000 7f1692ffd640 1 -- 172.21.15.163:0/625746159 <== osd.4 v2:172.21.15.192:6824/272496966 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f169c06ecb0 con 0x7f168007e470 2024-05-08T06:36:51.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.641+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/2559308876 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb58c0763d0 msgr2=0x7fb58c078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.641+0000 7fb5b1c1d640 1 --2- 172.21.15.163:0/2559308876 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb58c0763d0 0x7fb58c078890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb59c034000 tx=0x7fb59c02e000 comp rx=0 tx=0).stop 2024-05-08T06:36:51.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.641+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/2559308876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ac15f4c0 msgr2=0x7fb5ac205790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.641+0000 7fb5b1c1d640 1 --2- 172.21.15.163:0/2559308876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ac15f4c0 0x7fb5ac205790 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400efc0 tx=0x7fb5a400c490 comp rx=0 tx=0).stop 2024-05-08T06:36:51.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.641+0000 7fb5abfff640 1 -- 172.21.15.163:0/2559308876 reap_dead start 2024-05-08T06:36:51.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.641+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/2559308876 shutdown_connections 2024-05-08T06:36:51.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.641+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/2559308876 >> 172.21.15.163:0/2559308876 conn(0x7fb5ac06d300 msgr2=0x7fb5ac15fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.641+0000 7f1690ff9640 1 -- 172.21.15.163:0/625746159 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x7f168007e470 msgr2=0x7f1680080890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.641+0000 7f1690ff9640 1 --2- 172.21.15.163:0/625746159 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x7f168007e470 0x7f1680080890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.642+0000 7f83d1ffb640 1 -- 172.21.15.163:0/2704293457 --> [v2:172.21.15.192:6808/1747004047,v1:172.21.15.192:6809/1747004047] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83e4053520 con 0x7f83d807e620 2024-05-08T06:36:51.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.642+0000 7f1690ff9640 1 -- 172.21.15.163:0/625746159 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f16800762d0 msgr2=0x7f1680078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.642+0000 7f1690ff9640 1 --2- 172.21.15.163:0/625746159 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f16800762d0 0x7f1680078790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1698034000 tx=0x7f169802e000 comp rx=0 tx=0).stop 2024-05-08T06:36:51.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.642+0000 7f1690ff9640 1 -- 172.21.15.163:0/625746159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c1629e0 msgr2=0x7f169c1542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.642+0000 7f1690ff9640 1 --2- 172.21.15.163:0/625746159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c1629e0 0x7f169c1542e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f168c00b410 tx=0x7f168c00b8d0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.642+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/2559308876 shutdown_connections 2024-05-08T06:36:51.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.642+0000 7f16a2a0f640 1 -- 172.21.15.163:0/625746159 reap_dead start 2024-05-08T06:36:51.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.642+0000 7fb5b1c1d640 1 -- 172.21.15.163:0/2559308876 wait complete. 2024-05-08T06:36:51.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.642+0000 7f1690ff9640 1 -- 172.21.15.163:0/625746159 shutdown_connections 2024-05-08T06:36:51.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.642+0000 7f1690ff9640 1 -- 172.21.15.163:0/625746159 >> 172.21.15.163:0/625746159 conn(0x7f169c09e410 msgr2=0x7f169c06e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.723 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.642+0000 7f83d3fff640 1 -- 172.21.15.163:0/2704293457 <== osd.7 v2:172.21.15.192:6808/1747004047 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f83e4053520 con 0x7f83d807e620 2024-05-08T06:36:51.723 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.642+0000 7f1690ff9640 1 -- 172.21.15.163:0/625746159 shutdown_connections 2024-05-08T06:36:51.724 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.643+0000 7f83d1ffb640 1 -- 172.21.15.163:0/2704293457 >> [v2:172.21.15.192:6808/1747004047,v1:172.21.15.192:6809/1747004047] conn(0x7f83d807e620 msgr2=0x7f83d8080a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.724 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.642+0000 7f1690ff9640 1 -- 172.21.15.163:0/625746159 wait complete. 2024-05-08T06:36:51.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.643+0000 7f96ee7fc640 1 -- 172.21.15.163:0/3774413612 --> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9708166d60 con 0x7f96f407e5c0 2024-05-08T06:36:51.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.643+0000 7f83d1ffb640 1 --2- 172.21.15.163:0/2704293457 >> [v2:172.21.15.192:6808/1747004047,v1:172.21.15.192:6809/1747004047] conn(0x7f83d807e620 0x7f83d8080a40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.643+0000 7f83d1ffb640 1 -- 172.21.15.163:0/2704293457 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f83d80763a0 msgr2=0x7f83d8078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.643+0000 7f83d1ffb640 1 --2- 172.21.15.163:0/2704293457 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f83d80763a0 0x7f83d8078860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f83dc000f60 tx=0x7f83dc009040 comp rx=0 tx=0).stop 2024-05-08T06:36:51.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.643+0000 7f83d1ffb640 1 -- 172.21.15.163:0/2704293457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83e4008c20 msgr2=0x7f83e404f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.643+0000 7f83d1ffb640 1 --2- 172.21.15.163:0/2704293457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83e4008c20 0x7f83e404f720 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f83e000b480 tx=0x7f83e000b940 comp rx=0 tx=0).stop 2024-05-08T06:36:51.727 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.643+0000 7f83eb7fe640 1 -- 172.21.15.163:0/2704293457 reap_dead start 2024-05-08T06:36:51.727 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.643+0000 7f83d1ffb640 1 -- 172.21.15.163:0/2704293457 shutdown_connections 2024-05-08T06:36:51.727 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.643+0000 7f83d1ffb640 1 -- 172.21.15.163:0/2704293457 >> 172.21.15.163:0/2704293457 conn(0x7f83e4020040 msgr2=0x7f83e400d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.727 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.643+0000 7f5a8effd640 1 -- 172.21.15.163:0/2302273947 --> [v2:172.21.15.163:6800/3059241761,v1:172.21.15.163:6801/3059241761] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5a84002dd0 con 0x7f5a8807e8a0 2024-05-08T06:36:51.727 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.643+0000 7f9704ff9640 1 -- 172.21.15.163:0/3774413612 <== osd.6 v2:172.21.15.192:6816/1074108675 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9708166d60 con 0x7f96f407e5c0 2024-05-08T06:36:51.727 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.643+0000 7f5aa17fa640 1 -- 172.21.15.163:0/2302273947 <== osd.0 v2:172.21.15.163:6800/3059241761 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5a84002dd0 con 0x7f5a8807e8a0 2024-05-08T06:36:51.728 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.644+0000 7f96ee7fc640 1 -- 172.21.15.163:0/3774413612 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x7f96f407e5c0 msgr2=0x7f96f40809e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.728 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.644+0000 7f96ee7fc640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x7f96f407e5c0 0x7f96f40809e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.644+0000 7f96ee7fc640 1 -- 172.21.15.163:0/3774413612 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f96f40763a0 msgr2=0x7f96f4078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.644+0000 7f96ee7fc640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f96f40763a0 0x7f96f4078860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96f8007f30 tx=0x7f96f8007ec0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.644+0000 7f96ee7fc640 1 -- 172.21.15.163:0/3774413612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9708073cb0 msgr2=0x7f970813f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.644+0000 7f5a8effd640 1 -- 172.21.15.163:0/2302273947 >> [v2:172.21.15.163:6800/3059241761,v1:172.21.15.163:6801/3059241761] conn(0x7f5a8807e8a0 msgr2=0x7f5a88080cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.644+0000 7f5a8effd640 1 --2- 172.21.15.163:0/2302273947 >> [v2:172.21.15.163:6800/3059241761,v1:172.21.15.163:6801/3059241761] conn(0x7f5a8807e8a0 0x7f5a88080cc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.644+0000 7f83d1ffb640 1 -- 172.21.15.163:0/2704293457 shutdown_connections 2024-05-08T06:36:51.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.644+0000 7f83d1ffb640 1 -- 172.21.15.163:0/2704293457 wait complete. 2024-05-08T06:36:51.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.644+0000 7f5a8effd640 1 -- 172.21.15.163:0/2302273947 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5a88076700 msgr2=0x7f5a88078bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.644+0000 7f5a8effd640 1 --2- 172.21.15.163:0/2302273947 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5a88076700 0x7f5a88078bc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4065ed0 tx=0x7f5aa406ad80 comp rx=0 tx=0).stop 2024-05-08T06:36:51.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.644+0000 7f5a8effd640 1 -- 172.21.15.163:0/2302273947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a940052e0 msgr2=0x7f5a9c1784c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.644+0000 7f5a8effd640 1 --2- 172.21.15.163:0/2302273947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a940052e0 0x7f5a9c1784c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5a980108f0 tx=0x7f5a98010db0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.645+0000 7f5aa90c1640 1 -- 172.21.15.163:0/2302273947 reap_dead start 2024-05-08T06:36:51.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.644+0000 7f96ee7fc640 1 --2- 172.21.15.163:0/3774413612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9708073cb0 0x7f970813f160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00d5f0 tx=0x7f96fc00dab0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.645+0000 7fe1553c1640 1 -- 172.21.15.163:0/2067982890 --> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe130002dd0 con 0x7fe13c07e5c0 2024-05-08T06:36:51.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.645+0000 7f970c9ba640 1 -- 172.21.15.163:0/3774413612 reap_dead start 2024-05-08T06:36:51.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.645+0000 7f96ee7fc640 1 -- 172.21.15.163:0/3774413612 shutdown_connections 2024-05-08T06:36:51.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.645+0000 7f96ee7fc640 1 -- 172.21.15.163:0/3774413612 >> 172.21.15.163:0/3774413612 conn(0x7f97080b29b0 msgr2=0x7f970814b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.645+0000 7f5a8effd640 1 -- 172.21.15.163:0/2302273947 shutdown_connections 2024-05-08T06:36:51.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.645+0000 7f5a8effd640 1 -- 172.21.15.163:0/2302273947 >> 172.21.15.163:0/2302273947 conn(0x7f5a9c055c50 msgr2=0x7f5a9c03a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.645+0000 7f81b6eda640 1 -- 172.21.15.163:0/3192259881 --> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f81b006a550 con 0x7f819c07e5f0 2024-05-08T06:36:51.732 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.645+0000 7f96ee7fc640 1 -- 172.21.15.163:0/3774413612 shutdown_connections 2024-05-08T06:36:51.732 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.645+0000 7f96ee7fc640 1 -- 172.21.15.163:0/3774413612 wait complete. 2024-05-08T06:36:51.732 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.645+0000 7fe1377fe640 1 -- 172.21.15.163:0/2067982890 <== osd.5 v2:172.21.15.192:6800/2911825781 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe130002dd0 con 0x7fe13c07e5c0 2024-05-08T06:36:51.732 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.645+0000 7f5a8effd640 1 -- 172.21.15.163:0/2302273947 shutdown_connections 2024-05-08T06:36:51.733 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.646+0000 7f5a8effd640 1 -- 172.21.15.163:0/2302273947 wait complete. 2024-05-08T06:36:51.733 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.646+0000 7f81ad7fa640 1 -- 172.21.15.163:0/3192259881 <== osd.1 v2:172.21.15.163:6824/2322245857 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f81b006a550 con 0x7f819c07e5f0 2024-05-08T06:36:51.733 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.646+0000 7fe1553c1640 1 -- 172.21.15.163:0/2067982890 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x7fe13c07e5c0 msgr2=0x7fe13c0809e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.733 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.646+0000 7fe1553c1640 1 --2- 172.21.15.163:0/2067982890 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x7fe13c07e5c0 0x7fe13c0809e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.734 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.646+0000 7f81b6eda640 1 -- 172.21.15.163:0/3192259881 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x7f819c07e5f0 msgr2=0x7f819c080a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.734 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.646+0000 7fe1553c1640 1 -- 172.21.15.163:0/2067982890 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe13c0763a0 msgr2=0x7fe13c078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.734 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.646+0000 7fe1553c1640 1 --2- 172.21.15.163:0/2067982890 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe13c0763a0 0x7fe13c078860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe148034000 tx=0x7fe148009000 comp rx=0 tx=0).stop 2024-05-08T06:36:51.734 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.646+0000 7fe1553c1640 1 -- 172.21.15.163:0/2067982890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1500bab60 msgr2=0x7fe1500b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.734 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.646+0000 7fe1553c1640 1 --2- 172.21.15.163:0/2067982890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1500bab60 0x7fe1500b5930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe14000b410 tx=0x7fe14000b8d0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.734 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.646+0000 7fe14f7fe640 1 -- 172.21.15.163:0/2067982890 reap_dead start 2024-05-08T06:36:51.734 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.646+0000 7f81b6eda640 1 --2- 172.21.15.163:0/3192259881 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x7f819c07e5f0 0x7f819c080a10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.735 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.646+0000 7fe1553c1640 1 -- 172.21.15.163:0/2067982890 shutdown_connections 2024-05-08T06:36:51.735 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.646+0000 7fe1553c1640 1 -- 172.21.15.163:0/2067982890 >> 172.21.15.163:0/2067982890 conn(0x7fe15006d300 msgr2=0x7fe150161ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.736 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.647+0000 7f81b6eda640 1 -- 172.21.15.163:0/3192259881 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f819c0763d0 msgr2=0x7f819c078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.736 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.647+0000 7f81b6eda640 1 --2- 172.21.15.163:0/3192259881 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f819c0763d0 0x7f819c078890 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f81a8034000 tx=0x7f81a8033040 comp rx=0 tx=0).stop 2024-05-08T06:36:51.736 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.647+0000 7f81b6eda640 1 -- 172.21.15.163:0/3192259881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81b00d1270 msgr2=0x7f81b0161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.736 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.647+0000 7f81b6eda640 1 --2- 172.21.15.163:0/3192259881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81b00d1270 0x7f81b0161eb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f81a000e600 tx=0x7f81a000eac0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.736 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.647+0000 7f81b5477640 1 -- 172.21.15.163:0/3192259881 reap_dead start 2024-05-08T06:36:51.736 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.647+0000 7f81b6eda640 1 -- 172.21.15.163:0/3192259881 shutdown_connections 2024-05-08T06:36:51.736 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.647+0000 7f81b6eda640 1 -- 172.21.15.163:0/3192259881 >> 172.21.15.163:0/3192259881 conn(0x7f81b006c500 msgr2=0x7f81b006c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.736 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.648+0000 7f81b6eda640 1 -- 172.21.15.163:0/3192259881 shutdown_connections 2024-05-08T06:36:51.736 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.648+0000 7f81b6eda640 1 -- 172.21.15.163:0/3192259881 wait complete. 2024-05-08T06:36:51.737 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.648+0000 7fe1553c1640 1 -- 172.21.15.163:0/2067982890 shutdown_connections 2024-05-08T06:36:51.737 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.648+0000 7fe1553c1640 1 -- 172.21.15.163:0/2067982890 wait complete. 2024-05-08T06:36:51.737 INFO:teuthology.orchestra.run.smithi163.stdout:55834574856 2024-05-08T06:36:51.738 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T06:36:51.739 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2024-05-08T06:36:51.739 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T06:36:51.740 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2024-05-08T06:36:51.740 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T06:36:51.740 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2024-05-08T06:36:51.741 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T06:36:51.742 INFO:teuthology.orchestra.run.smithi163.stdout:55834574857 2024-05-08T06:36:51.742 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T06:36:51.743 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2024-05-08T06:36:51.743 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T06:36:51.744 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2024-05-08T06:36:51.744 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T06:36:51.745 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2024-05-08T06:36:51.745 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T06:36:51.949 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.949+0000 7f589af5f640 1 Processor -- start 2024-05-08T06:36:51.949 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.949+0000 7f8e6af7b640 1 Processor -- start 2024-05-08T06:36:51.950 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.949+0000 7f589af5f640 1 -- start start 2024-05-08T06:36:51.950 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.951+0000 7f8a017c1640 1 Processor -- start 2024-05-08T06:36:51.950 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.951+0000 7f8a017c1640 1 -- start start 2024-05-08T06:36:51.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.951+0000 7f589af5f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f589415e690 0x7f5894072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.950+0000 7fa170ada640 1 Processor -- start 2024-05-08T06:36:51.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.950+0000 7fa170ada640 1 -- start start 2024-05-08T06:36:51.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.950+0000 7f8e6af7b640 1 -- start start 2024-05-08T06:36:51.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.951+0000 7f589af5f640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58940735c0 con 0x7f589406f3e0 2024-05-08T06:36:51.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.951+0000 7f589af5f640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58940b50e0 con 0x7f58940b4970 2024-05-08T06:36:51.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.951+0000 7f589af5f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f58940b53c0 con 0x7f589415e690 2024-05-08T06:36:51.953 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.951+0000 7f589975c640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f58940b4970 0x7f58940cd720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:57314/0 (socket says 172.21.15.163:57314) 2024-05-08T06:36:51.953 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.951+0000 7f589975c640 1 -- 172.21.15.163:0/65138480 learned_addr learned my addr 172.21.15.163:0/65138480 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:51.953 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2555164856 0 0) 0x7f58940b50e0 con 0x7f58940b4970 2024-05-08T06:36:51.953 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5880003520 con 0x7f58940b4970 2024-05-08T06:36:51.953 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2951532155 0 0) 0x7f58940735c0 con 0x7f589406f3e0 2024-05-08T06:36:51.953 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58940b50e0 con 0x7f589406f3e0 2024-05-08T06:36:51.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7fa170ada640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa16c0b4cb0 0x7fa16c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7fa170ada640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa16c0d1aa0 0x7fa16c15f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7f8e6af7b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8e5c050190 0x7f8e5c038070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7f8e6af7b640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e5c0544b0 con 0x7f8e5c00c090 2024-05-08T06:36:51.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7f8e6af7b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e5c0541d0 con 0x7f8e5c053c60 2024-05-08T06:36:51.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7f8e6af7b640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f8e5c005f20 con 0x7f8e5c050190 2024-05-08T06:36:51.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7fa169d9b640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fa16c073ed0 0x7fa16c0d1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:57322/0 (socket says 172.21.15.163:57322) 2024-05-08T06:36:51.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7f8e69f79640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8e5c00c090 0x7f8e5c0df180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57066/0 (socket says 172.21.15.163:57066) 2024-05-08T06:36:51.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7f8e69f79640 1 -- 172.21.15.163:0/68500582 learned_addr learned my addr 172.21.15.163:0/68500582 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:51.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7fa169d9b640 1 -- 172.21.15.163:0/3597189541 learned_addr learned my addr 172.21.15.163:0/3597189541 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:51.958 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7fd3df59e640 1 Processor -- start 2024-05-08T06:36:51.958 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7fd3df59e640 1 -- start start 2024-05-08T06:36:51.959 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.952+0000 7fa16ad9d640 1 --2- 172.21.15.163:0/3597189541 >> v2:172.21.15.192:3300/0 conn(0x7fa16c0d1aa0 0x7fa16c15f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.959 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7fa170ada640 1 -- 172.21.15.163:0/3597189541 --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa16c0726e0 con 0x7fa16c073ed0 2024-05-08T06:36:51.960 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f8a017c1640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f89f40543d0 0x7f89f40587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7fa16a59c640 1 --2- 172.21.15.163:0/3597189541 >> v2:172.21.15.163:3300/0 conn(0x7fa16c0b4cb0 0x7fa16c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7fa170ada640 1 -- 172.21.15.163:0/3597189541 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa16c0b5090 con 0x7fa16c0b4cb0 2024-05-08T06:36:51.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7fa170ada640 1 -- 172.21.15.163:0/3597189541 --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa16c06e590 con 0x7fa16c0d1aa0 2024-05-08T06:36:51.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7fa16959a640 1 -- 172.21.15.163:0/3597189541 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3636984617 0 0) 0x7fa16c0726e0 con 0x7fa16c073ed0 2024-05-08T06:36:51.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f8a017c1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89f40e6b80 con 0x7f89f403e880 2024-05-08T06:36:51.962 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7fa16959a640 1 -- 172.21.15.163:0/3597189541 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa154003520 con 0x7fa16c073ed0 2024-05-08T06:36:51.962 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 640889433 0 0) 0x7f58940b50e0 con 0x7f589406f3e0 2024-05-08T06:36:51.963 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f589415e690 msgr2=0x7f5894072d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:36:51.963 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f5898f5b640 1 --2- 172.21.15.163:0/65138480 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f589415e690 0x7f5894072d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.963 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 >> v1:172.21.15.192:6789/0 conn(0x7f58940b4970 legacy=0x7f58940cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.963 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58940b56a0 con 0x7f589406f3e0 2024-05-08T06:36:51.963 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f588c001a50 con 0x7f589406f3e0 2024-05-08T06:36:51.963 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 >> v1:172.21.15.163:6789/0 conn(0x7f589406f3e0 legacy=0x7f5894073ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.963 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f5898f5b640 1 --2- 172.21.15.163:0/65138480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5880004df0 0x7f58800037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.963 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f5898f5b640 1 --2- 172.21.15.163:0/65138480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5880003dc0 0x7f5880004240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.963 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f5898f5b640 1 --2- 172.21.15.163:0/65138480 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58800094b0 0x7f5880009930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.963 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f58940b53c0 con 0x7f5880004df0 2024-05-08T06:36:51.963 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f588000a1c0 con 0x7f5880003dc0 2024-05-08T06:36:51.963 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f588000a3c0 con 0x7f58800094b0 2024-05-08T06:36:51.964 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f8a017c1640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89f403ed10 con 0x7f89f40e67a0 2024-05-08T06:36:51.964 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f8a017c1640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f89f40573a0 con 0x7f89f40543d0 2024-05-08T06:36:51.964 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f89faffd640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f89f40e67a0 0x7f89f4053cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:57334/0 (socket says 172.21.15.163:57334) 2024-05-08T06:36:51.964 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f89faffd640 1 -- 172.21.15.163:0/2366845053 learned_addr learned my addr 172.21.15.163:0/2366845053 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:51.964 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f89f9ffb640 1 -- 172.21.15.163:0/2366845053 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3024475345 0 0) 0x7f89f403ed10 con 0x7f89f40e67a0 2024-05-08T06:36:51.964 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f89f9ffb640 1 -- 172.21.15.163:0/2366845053 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89e4003520 con 0x7f89f40e67a0 2024-05-08T06:36:51.964 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f89fb7fe640 1 --2- 172.21.15.163:0/2366845053 >> v2:172.21.15.192:3300/0 conn(0x7f89f40543d0 0x7f89f40587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.964 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7fa16959a640 1 -- 172.21.15.163:0/3597189541 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1453421639 0 0) 0x7fa154003520 con 0x7fa16c073ed0 2024-05-08T06:36:51.965 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7fa16959a640 1 -- 172.21.15.163:0/3597189541 >> v2:172.21.15.192:3300/0 conn(0x7fa16c0d1aa0 msgr2=0x7fa16c15f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.965 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7fa16959a640 1 --2- 172.21.15.163:0/3597189541 >> v2:172.21.15.192:3300/0 conn(0x7fa16c0d1aa0 0x7fa16c15f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.965 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7fa16959a640 1 -- 172.21.15.163:0/3597189541 >> v2:172.21.15.163:3300/0 conn(0x7fa16c0b4cb0 msgr2=0x7fa16c15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.965 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7fa16959a640 1 --2- 172.21.15.163:0/3597189541 >> v2:172.21.15.163:3300/0 conn(0x7fa16c0b4cb0 0x7fa16c15b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.965 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7fa16959a640 1 -- 172.21.15.163:0/3597189541 --> v1:172.21.15.192:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16c06e870 con 0x7fa16c073ed0 2024-05-08T06:36:51.965 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f8e68f77640 1 -- 172.21.15.163:0/68500582 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3287117550 0 0) 0x7f8e5c0541d0 con 0x7f8e5c053c60 2024-05-08T06:36:51.965 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7fa16959a640 1 -- 172.21.15.163:0/3597189541 <== mon.1 v1:172.21.15.192:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7fa1600016f0 con 0x7fa16c073ed0 2024-05-08T06:36:51.966 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f8e68f77640 1 -- 172.21.15.163:0/68500582 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e50003520 con 0x7f8e5c053c60 2024-05-08T06:36:51.966 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f589a75e640 1 --2- 172.21.15.163:0/65138480 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58800094b0 0x7f5880009930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.966 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f89f9ffb640 1 -- 172.21.15.163:0/2366845053 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 690521149 0 0) 0x7f89e4003520 con 0x7f89f40e67a0 2024-05-08T06:36:51.967 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f89f9ffb640 1 -- 172.21.15.163:0/2366845053 >> v2:172.21.15.192:3300/0 conn(0x7f89f40543d0 msgr2=0x7f89f40587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.967 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.953+0000 7f89f9ffb640 1 --2- 172.21.15.163:0/2366845053 >> v2:172.21.15.192:3300/0 conn(0x7f89f40543d0 0x7f89f40587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.967 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f89f9ffb640 1 -- 172.21.15.163:0/2366845053 >> v1:172.21.15.163:6789/0 conn(0x7f89f403e880 legacy=0x7f89f40580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.967 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f89f9ffb640 1 -- 172.21.15.163:0/2366845053 --> v1:172.21.15.192:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f40575f0 con 0x7f89f40e67a0 2024-05-08T06:36:51.967 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fa16959a640 1 -- 172.21.15.163:0/3597189541 >> v1:172.21.15.192:6789/0 conn(0x7fa16c073ed0 legacy=0x7fa16c0d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.967 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f89f9ffb640 1 -- 172.21.15.163:0/2366845053 <== mon.1 v1:172.21.15.192:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f89ec0016d0 con 0x7f89f40e67a0 2024-05-08T06:36:51.967 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f89f9ffb640 1 -- 172.21.15.163:0/2366845053 >> v1:172.21.15.192:6789/0 conn(0x7f89f40e67a0 legacy=0x7f89f4053cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.968 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f589975c640 1 --2- 172.21.15.163:0/65138480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5880003dc0 0x7f5880004240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.968 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fa16959a640 1 --2- 172.21.15.163:0/3597189541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154003a50 0x7fa154003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.968 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fa16959a640 1 --2- 172.21.15.163:0/3597189541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa154004800 0x7fa154004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.969 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f89f9ffb640 1 --2- 172.21.15.163:0/2366845053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89e4003a40 0x7f89e4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.969 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f89f9ffb640 1 --2- 172.21.15.163:0/2366845053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89e4004800 0x7f89e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.969 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f89f9ffb640 1 --2- 172.21.15.163:0/2366845053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89e40053c0 0x7f89e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.969 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f89f9ffb640 1 -- 172.21.15.163:0/2366845053 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f89f40573a0 con 0x7f89e40053c0 2024-05-08T06:36:51.969 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f89f9ffb640 1 -- 172.21.15.163:0/2366845053 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f89e400c2e0 con 0x7f89e4003a40 2024-05-08T06:36:51.969 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f89f9ffb640 1 -- 172.21.15.163:0/2366845053 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f89e400c4e0 con 0x7f89e4004800 2024-05-08T06:36:51.969 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fa16959a640 1 --2- 172.21.15.163:0/3597189541 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1540053c0 0x7fa15400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.969 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f8e68f77640 1 -- 172.21.15.163:0/68500582 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4175802688 0 0) 0x7f8e50003520 con 0x7f8e5c053c60 2024-05-08T06:36:51.970 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fd3df59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d80eacf0 0x7fd3d804fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.970 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fa16959a640 1 -- 172.21.15.163:0/3597189541 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c06e590 con 0x7fa154003a50 2024-05-08T06:36:51.970 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fa16959a640 1 -- 172.21.15.163:0/3597189541 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa15400c250 con 0x7fa154004800 2024-05-08T06:36:51.970 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fa16959a640 1 -- 172.21.15.163:0/3597189541 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa15400c450 con 0x7fa1540053c0 2024-05-08T06:36:51.971 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fa169d9b640 1 --2- 172.21.15.163:0/3597189541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa154004800 0x7fa154004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.971 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fa16ad9d640 1 --2- 172.21.15.163:0/3597189541 >> v2:172.21.15.192:3300/0 conn(0x7fa16c0d1aa0 0x7fa16c15f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:51.971 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f89faffd640 1 --2- 172.21.15.163:0/2366845053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89e4003a40 0x7f89e4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.971 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f8e6a77a640 1 --2- 172.21.15.163:0/68500582 >> v2:172.21.15.163:3300/0 conn(0x7f8e5c050190 0x7f8e5c038070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.972 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f8e68f77640 1 -- 172.21.15.163:0/68500582 >> v2:172.21.15.163:3300/0 conn(0x7f8e5c050190 msgr2=0x7f8e5c038070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.972 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f8e68f77640 1 --2- 172.21.15.163:0/68500582 >> v2:172.21.15.163:3300/0 conn(0x7f8e5c050190 0x7f8e5c038070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.972 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f8e68f77640 1 -- 172.21.15.163:0/68500582 >> v1:172.21.15.163:6789/0 conn(0x7f8e5c00c090 legacy=0x7f8e5c0df180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.972 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f8e68f77640 1 -- 172.21.15.163:0/68500582 --> v1:172.21.15.192:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5c006200 con 0x7f8e5c053c60 2024-05-08T06:36:51.972 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fd3de59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d80eacf0 0x7fd3d804fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.972 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fd3de59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d80eacf0 0x7fd3d804fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:48298/0 (socket says 172.21.15.163:48298) 2024-05-08T06:36:51.972 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fd3de59c640 1 -- 172.21.15.163:0/3241988472 learned_addr learned my addr 172.21.15.163:0/3241988472 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.972 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fd3df59e640 1 -- 172.21.15.163:0/3241988472 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3d8053710 con 0x7fd3d8006e90 2024-05-08T06:36:51.973 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fa16a59c640 1 --2- 172.21.15.163:0/3597189541 >> v2:172.21.15.163:3300/0 conn(0x7fa16c0b4cb0 0x7fa16c15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:51.973 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f8e6a77a640 1 --2- 172.21.15.163:0/68500582 >> v2:172.21.15.163:3300/0 conn(0x7f8e5c050190 0x7f8e5c038070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:51.973 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f8e68f77640 1 -- 172.21.15.163:0/68500582 <== mon.1 v1:172.21.15.192:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f8e540016f0 con 0x7f8e5c053c60 2024-05-08T06:36:51.973 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f89fb7fe640 1 --2- 172.21.15.163:0/2366845053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89e40053c0 0x7f89e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fd3df59e640 1 -- 172.21.15.163:0/3241988472 --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3d80539f0 con 0x7fd3d804ffc0 2024-05-08T06:36:51.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f8e68f77640 1 -- 172.21.15.163:0/68500582 >> v1:172.21.15.192:6789/0 conn(0x7f8e5c053c60 legacy=0x7f8e5c04fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7fd3df59e640 1 -- 172.21.15.163:0/3241988472 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d8054320 con 0x7fd3d80eacf0 2024-05-08T06:36:51.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f89fa7fc640 1 --2- 172.21.15.163:0/2366845053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89e4004800 0x7f89e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.975 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f8e68f77640 1 --2- 172.21.15.163:0/68500582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50003a00 0x7f8e50003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.975 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f8e68f77640 1 --2- 172.21.15.163:0/68500582 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e50004800 0x7f8e50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.975 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f589975c640 1 -- 172.21.15.163:0/65138480 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58800094b0 msgr2=0x7f5880009930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.975 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f589975c640 1 --2- 172.21.15.163:0/65138480 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58800094b0 0x7f5880009930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.975 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f589975c640 1 -- 172.21.15.163:0/65138480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5880004df0 msgr2=0x7f58800037a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T06:36:51.976 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f589975c640 1 --2- 172.21.15.163:0/65138480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5880004df0 0x7f58800037a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.976 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f589975c640 1 -- 172.21.15.163:0/65138480 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58940b56a0 con 0x7f5880003dc0 2024-05-08T06:36:51.976 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f8e68f77640 1 --2- 172.21.15.163:0/68500582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e50005290 0x7f8e5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.976 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.954+0000 7f89faffd640 1 -- 172.21.15.163:0/2366845053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89e4004800 msgr2=0x7f89e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.976 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7f89faffd640 1 --2- 172.21.15.163:0/2366845053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89e4004800 0x7f89e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7f8e68f77640 1 -- 172.21.15.163:0/68500582 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c005f20 con 0x7f8e50005290 2024-05-08T06:36:51.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7f8e68f77640 1 -- 172.21.15.163:0/68500582 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5000c1e0 con 0x7f8e50003a00 2024-05-08T06:36:51.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7f8e68f77640 1 -- 172.21.15.163:0/68500582 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8e5000c3e0 con 0x7f8e50004800 2024-05-08T06:36:51.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7f89faffd640 1 -- 172.21.15.163:0/2366845053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89e40053c0 msgr2=0x7f89e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7f89faffd640 1 --2- 172.21.15.163:0/2366845053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89e40053c0 0x7f89e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7f89faffd640 1 -- 172.21.15.163:0/2366845053 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f40575f0 con 0x7f89e4003a40 2024-05-08T06:36:51.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7fa169d9b640 1 -- 172.21.15.163:0/3597189541 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1540053c0 msgr2=0x7fa15400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7fa169d9b640 1 --2- 172.21.15.163:0/3597189541 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1540053c0 0x7fa15400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7fa169d9b640 1 -- 172.21.15.163:0/3597189541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154003a50 msgr2=0x7fa154003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7fa169d9b640 1 --2- 172.21.15.163:0/3597189541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154003a50 0x7fa154003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7fa169d9b640 1 -- 172.21.15.163:0/3597189541 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa154003520 con 0x7fa154004800 2024-05-08T06:36:51.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7fa16ad9d640 1 -- 172.21.15.163:0/3597189541 reap_dead start 2024-05-08T06:36:51.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7fd3de59c640 1 -- 172.21.15.163:0/3241988472 >> v1:172.21.15.192:6789/0 conn(0x7fd3d804ffc0 legacy=0x7fd3d8037bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7f8e6a77a640 1 --2- 172.21.15.163:0/68500582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e50005290 0x7f8e5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7fd3de59c640 1 -- 172.21.15.163:0/3241988472 >> v1:172.21.15.163:6789/0 conn(0x7fd3d8006e90 legacy=0x7fd3d80382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.955+0000 7fd3de59c640 1 -- 172.21.15.163:0/3241988472 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d8053cd0 con 0x7fd3d80eacf0 2024-05-08T06:36:51.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f8e6a77a640 1 -- 172.21.15.163:0/68500582 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e50004800 msgr2=0x7f8e50004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:36:51.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f8e6a77a640 1 --2- 172.21.15.163:0/68500582 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e50004800 0x7f8e50004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f8e6a77a640 1 -- 172.21.15.163:0/68500582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50003a00 msgr2=0x7f8e50003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f8e6a77a640 1 --2- 172.21.15.163:0/68500582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50003a00 0x7f8e50003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f8e6a77a640 1 -- 172.21.15.163:0/68500582 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e50003520 con 0x7f8e50005290 2024-05-08T06:36:51.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f89fb7fe640 1 --2- 172.21.15.163:0/2366845053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89e40053c0 0x7f89e400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:51.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f589975c640 1 --2- 172.21.15.163:0/65138480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5880003dc0 0x7f5880004240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5884001a70 tx=0x7f5880004820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9ff020de1c721bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f89fb7fe640 1 -- 172.21.15.163:0/2366845053 reap_dead start 2024-05-08T06:36:51.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f89fa7fc640 1 --2- 172.21.15.163:0/2366845053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89e4004800 0x7f89e4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:51.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f89faffd640 1 --2- 172.21.15.163:0/2366845053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89e4003a40 0x7f89e4003e80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c270 tx=0x7f89e4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfb605984f7e1d8b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884001690 con 0x7f5880003dc0 2024-05-08T06:36:51.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588402e070 con 0x7f5880003dc0 2024-05-08T06:36:51.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f5898f5b640 1 -- 172.21.15.163:0/65138480 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58840048a0 con 0x7f5880003dc0 2024-05-08T06:36:51.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7fa169d9b640 1 --2- 172.21.15.163:0/3597189541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa154004800 0x7fa154004c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa160005730 tx=0x7fa154005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57d94d501761a9d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7fa16959a640 1 -- 172.21.15.163:0/3597189541 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa160005430 con 0x7fa154004800 2024-05-08T06:36:51.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7fa16959a640 1 -- 172.21.15.163:0/3597189541 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa160038640 con 0x7fa154004800 2024-05-08T06:36:51.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7fa16959a640 1 -- 172.21.15.163:0/3597189541 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa160038af0 con 0x7fa154004800 2024-05-08T06:36:51.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f8e6a77a640 1 -- 172.21.15.163:0/68500582 reap_dead start 2024-05-08T06:36:51.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7fd3de59c640 1 --2- 172.21.15.163:0/3241988472 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d80eacf0 0x7fd3d804fa80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00098a0 tx=0x7fd3d002a2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac46c82a84e659b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.983 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f324e704640 1 Processor -- start 2024-05-08T06:36:51.983 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7ffb88cc8640 1 Processor -- start 2024-05-08T06:36:51.984 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f89f9ffb640 1 -- 172.21.15.163:0/2366845053 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec036070 con 0x7f89e4003a40 2024-05-08T06:36:51.984 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f89f9ffb640 1 -- 172.21.15.163:0/2366845053 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ec005ad0 con 0x7f89e4003a40 2024-05-08T06:36:51.984 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7ffb88cc8640 1 -- start start 2024-05-08T06:36:51.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f89f9ffb640 1 -- 172.21.15.163:0/2366845053 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec036070 con 0x7f89e4003a40 2024-05-08T06:36:51.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f589af5f640 1 -- 172.21.15.163:0/65138480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5880003dc0 msgr2=0x7f5880004240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f589af5f640 1 --2- 172.21.15.163:0/65138480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5880003dc0 0x7f5880004240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5884001a70 tx=0x7f5880004820 comp rx=0 tx=0).stop 2024-05-08T06:36:51.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f589a75e640 1 -- 172.21.15.163:0/65138480 reap_dead start 2024-05-08T06:36:51.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f8e6a77a640 1 --2- 172.21.15.163:0/68500582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e50005290 0x7f8e5000b9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8e64050c60 tx=0x7f8e50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb26c87391e2bc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:51.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f8e68f77640 1 -- 172.21.15.163:0/68500582 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64050cb0 con 0x7f8e50005290 2024-05-08T06:36:51.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f8e68f77640 1 -- 172.21.15.163:0/68500582 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e6406f410 con 0x7f8e50005290 2024-05-08T06:36:51.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f8e68f77640 1 -- 172.21.15.163:0/68500582 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6406f7c0 con 0x7f8e50005290 2024-05-08T06:36:51.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7fd3dd59a640 1 -- 172.21.15.163:0/3241988472 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0037070 con 0x7fd3d80eacf0 2024-05-08T06:36:51.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7fd3dd59a640 1 -- 172.21.15.163:0/3241988472 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d0032640 con 0x7fd3d80eacf0 2024-05-08T06:36:51.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7fd3dd59a640 1 -- 172.21.15.163:0/3241988472 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0032a10 con 0x7fd3d80eacf0 2024-05-08T06:36:51.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7fce4007b640 1 Processor -- start 2024-05-08T06:36:51.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7fce4007b640 1 -- start start 2024-05-08T06:36:51.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.956+0000 7f324e704640 1 -- start start 2024-05-08T06:36:51.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f324e704640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3248166d90 0x7f32480b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f324e704640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32480b4460 0x7f32480d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f324e704640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32480b49c0 con 0x7f32480d17a0 2024-05-08T06:36:51.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f324e704640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f32480adf00 con 0x7f3248166d90 2024-05-08T06:36:51.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.957+0000 7f324e704640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f32480ae150 con 0x7f32480b4460 2024-05-08T06:36:51.989 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7fa170ada640 1 -- 172.21.15.163:0/3597189541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa154004800 msgr2=0x7fa154004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.989 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7fa170ada640 1 --2- 172.21.15.163:0/3597189541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa154004800 0x7fa154004c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa160005730 tx=0x7fa154005830 comp rx=0 tx=0).stop 2024-05-08T06:36:51.989 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f3247fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3248166d90 0x7f32480b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f8a017c1640 1 -- 172.21.15.163:0/2366845053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89e4003a40 msgr2=0x7f89e4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f8a017c1640 1 --2- 172.21.15.163:0/2366845053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89e4003a40 0x7f89e4003e80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c270 tx=0x7f89e4005830 comp rx=0 tx=0).stop 2024-05-08T06:36:51.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f3247fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3248166d90 0x7f32480b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35746/0 (socket says 172.21.15.163:35746) 2024-05-08T06:36:51.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f3247fff640 1 -- 172.21.15.163:0/1715685369 learned_addr learned my addr 172.21.15.163:0/1715685369 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:51.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f8e6af7b640 1 -- 172.21.15.163:0/68500582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e50005290 msgr2=0x7f8e5000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f589af5f640 1 -- 172.21.15.163:0/65138480 shutdown_connections 2024-05-08T06:36:51.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7fa170ada640 1 -- 172.21.15.163:0/3597189541 shutdown_connections 2024-05-08T06:36:51.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7fa170ada640 1 --2- 172.21.15.163:0/3597189541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa154004800 0x7fa154004c80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f589af5f640 1 -- 172.21.15.163:0/65138480 >> 172.21.15.163:0/65138480 conn(0x7f58940b29d0 msgr2=0x7f58940b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.992 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7fa170ada640 1 -- 172.21.15.163:0/3597189541 >> 172.21.15.163:0/3597189541 conn(0x7fa16c06c740 msgr2=0x7fa16c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.992 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f8e6af7b640 1 --2- 172.21.15.163:0/68500582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e50005290 0x7f8e5000b9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8e64050c60 tx=0x7f8e50005830 comp rx=0 tx=0).stop 2024-05-08T06:36:51.992 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f8e6af7b640 1 -- 172.21.15.163:0/68500582 shutdown_connections 2024-05-08T06:36:51.992 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7fd3df59e640 1 -- 172.21.15.163:0/3241988472 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d80eacf0 msgr2=0x7fd3d804fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:51.993 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f8a017c1640 1 -- 172.21.15.163:0/2366845053 shutdown_connections 2024-05-08T06:36:51.993 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f8a017c1640 1 --2- 172.21.15.163:0/2366845053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89e4003a40 0x7f89e4003e80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.993 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f8e6af7b640 1 --2- 172.21.15.163:0/68500582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e50005290 0x7f8e5000b9c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.993 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f8e6af7b640 1 -- 172.21.15.163:0/68500582 >> 172.21.15.163:0/68500582 conn(0x7f8e5c009fa0 msgr2=0x7f8e5c00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.993 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f8a017c1640 1 -- 172.21.15.163:0/2366845053 >> 172.21.15.163:0/2366845053 conn(0x7f89f4055990 msgr2=0x7f89f4055d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:51.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f3247fff640 1 -- 172.21.15.163:0/1715685369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32480b4460 msgr2=0x7f32480d1260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:36:51.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f3247fff640 1 --2- 172.21.15.163:0/1715685369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32480b4460 0x7f32480d1260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f3247fff640 1 -- 172.21.15.163:0/1715685369 >> v1:172.21.15.163:6789/0 conn(0x7f32480d17a0 legacy=0x7f3248167330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f3247fff640 1 -- 172.21.15.163:0/1715685369 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32480ae430 con 0x7f3248166d90 2024-05-08T06:36:51.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7fd3df59e640 1 --2- 172.21.15.163:0/3241988472 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d80eacf0 0x7fd3d804fa80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00098a0 tx=0x7fd3d002a2e0 comp rx=0 tx=0).stop 2024-05-08T06:36:51.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7ffb88cc8640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffb840d1260 0x7ffb840d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7ffb88cc8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb840b6160 0x7ffb8415ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7ffb88cc8640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb840d1b80 con 0x7ffb8415d3d0 2024-05-08T06:36:51.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f8e6af7b640 1 -- 172.21.15.163:0/68500582 shutdown_connections 2024-05-08T06:36:51.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7ffb88cc8640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7ffb840aff30 con 0x7ffb840d1260 2024-05-08T06:36:51.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7ffb88cc8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffb840b0210 con 0x7ffb840b6160 2024-05-08T06:36:51.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7f8e6af7b640 1 -- 172.21.15.163:0/68500582 wait complete. 2024-05-08T06:36:51.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7ffb81d9b640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ffb8415d3d0 0x7ffb8406e380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57108/0 (socket says 172.21.15.163:57108) 2024-05-08T06:36:51.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7ffb81d9b640 1 -- 172.21.15.163:0/2576389670 learned_addr learned my addr 172.21.15.163:0/2576389670 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:51.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.958+0000 7ffb82d9d640 1 --2- 172.21.15.163:0/2576389670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb840b6160 0x7ffb8415ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7ffb8259c640 1 --2- 172.21.15.163:0/2576389670 >> v2:172.21.15.163:3300/0 conn(0x7ffb840d1260 0x7ffb840d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7ffb8159a640 1 -- 172.21.15.163:0/2576389670 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2796992765 0 0) 0x7ffb840d1b80 con 0x7ffb8415d3d0 2024-05-08T06:36:51.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fce4007b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fce38163ad0 0x7fce38164820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:51.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7ffb8159a640 1 -- 172.21.15.163:0/2576389670 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb6c003520 con 0x7ffb8415d3d0 2024-05-08T06:36:51.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7ffb82d9d640 1 -- 172.21.15.163:0/2576389670 >> v2:172.21.15.163:3300/0 conn(0x7ffb840d1260 msgr2=0x7ffb840d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:51.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7f8a017c1640 1 -- 172.21.15.163:0/2366845053 shutdown_connections 2024-05-08T06:36:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fce3de17640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fce380cf8f0 0x7fce380cfcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57118/0 (socket says 172.21.15.163:57118) 2024-05-08T06:36:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fce3de17640 1 -- 172.21.15.163:0/3788688554 learned_addr learned my addr 172.21.15.163:0/3788688554 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fce4007b640 1 -- 172.21.15.163:0/3788688554 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce38167e50 con 0x7fce380cf8f0 2024-05-08T06:36:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7f8a017c1640 1 -- 172.21.15.163:0/2366845053 wait complete. 2024-05-08T06:36:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fa170ada640 1 -- 172.21.15.163:0/3597189541 shutdown_connections 2024-05-08T06:36:51.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fce3d616640 1 --2- 172.21.15.163:0/3788688554 >> v2:172.21.15.192:3300/0 conn(0x7fce38163ad0 0x7fce38164820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:51.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fce4007b640 1 -- 172.21.15.163:0/3788688554 --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce38168130 con 0x7fce38167470 2024-05-08T06:36:51.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fce4007b640 1 -- 172.21.15.163:0/3788688554 --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fce38168410 con 0x7fce38163ad0 2024-05-08T06:36:51.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7ffb8159a640 1 -- 172.21.15.163:0/2576389670 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1261105364 0 0) 0x7ffb6c003520 con 0x7ffb8415d3d0 2024-05-08T06:36:51.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7f589af5f640 1 -- 172.21.15.163:0/65138480 shutdown_connections 2024-05-08T06:36:52.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7f589af5f640 1 -- 172.21.15.163:0/65138480 wait complete. 2024-05-08T06:36:52.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fa170ada640 1 -- 172.21.15.163:0/3597189541 wait complete. 2024-05-08T06:36:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.960+0000 7fd3df59e640 1 -- 172.21.15.163:0/3241988472 shutdown_connections 2024-05-08T06:36:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.960+0000 7fd3df59e640 1 --2- 172.21.15.163:0/3241988472 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d80eacf0 0x7fd3d804fa80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fce3ce15640 1 -- 172.21.15.163:0/3788688554 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2331896054 0 0) 0x7fce38167e50 con 0x7fce380cf8f0 2024-05-08T06:36:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fce3ce15640 1 -- 172.21.15.163:0/3788688554 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce24003520 con 0x7fce380cf8f0 2024-05-08T06:36:52.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fce3ce15640 1 -- 172.21.15.163:0/3788688554 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 580722809 0 0) 0x7fce38168130 con 0x7fce38167470 2024-05-08T06:36:52.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fce3ce15640 1 -- 172.21.15.163:0/3788688554 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce38167e50 con 0x7fce38167470 2024-05-08T06:36:52.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fce3d616640 1 -- 172.21.15.163:0/3788688554 >> v1:172.21.15.192:6789/0 conn(0x7fce38167470 legacy=0x7fce380d3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fce3d616640 1 -- 172.21.15.163:0/3788688554 >> v1:172.21.15.163:6789/0 conn(0x7fce380cf8f0 legacy=0x7fce380cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7fce3d616640 1 -- 172.21.15.163:0/3788688554 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce38164020 con 0x7fce38163ad0 2024-05-08T06:36:52.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7f3247fff640 1 --2- 172.21.15.163:0/1715685369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3248166d90 0x7f32480b3f20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3240009e50 tx=0x7f324002a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b296286e762e3b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.960+0000 7fd3df59e640 1 -- 172.21.15.163:0/3241988472 >> 172.21.15.163:0/3241988472 conn(0x7fd3d8009fa0 msgr2=0x7fd3d800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:52.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7ffb82d9d640 1 --2- 172.21.15.163:0/2576389670 >> v2:172.21.15.163:3300/0 conn(0x7ffb840d1260 0x7ffb840d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7ffb82d9d640 1 -- 172.21.15.163:0/2576389670 >> v1:172.21.15.163:6789/0 conn(0x7ffb8415d3d0 legacy=0x7ffb8406e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.959+0000 7ffb82d9d640 1 -- 172.21.15.163:0/2576389670 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb840b04f0 con 0x7ffb840b6160 2024-05-08T06:36:52.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.960+0000 7ffb8259c640 1 --2- 172.21.15.163:0/2576389670 >> v2:172.21.15.163:3300/0 conn(0x7ffb840d1260 0x7ffb840d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:52.004 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.960+0000 7f3246ffd640 1 -- 172.21.15.163:0/1715685369 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240036070 con 0x7f3248166d90 2024-05-08T06:36:52.004 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.960+0000 7f3246ffd640 1 -- 172.21.15.163:0/1715685369 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3240031600 con 0x7f3248166d90 2024-05-08T06:36:52.004 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.960+0000 7f3246ffd640 1 -- 172.21.15.163:0/1715685369 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32400319d0 con 0x7f3248166d90 2024-05-08T06:36:52.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.960+0000 7f8e6af7b640 1 Processor -- start 2024-05-08T06:36:52.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.960+0000 7f8e6af7b640 1 -- start start 2024-05-08T06:36:52.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.960+0000 7fd3df59e640 1 -- 172.21.15.163:0/3241988472 shutdown_connections 2024-05-08T06:36:52.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.960+0000 7ffb82d9d640 1 --2- 172.21.15.163:0/2576389670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb840b6160 0x7ffb8415ddd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009d90 tx=0x7ffb7c02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4728bc646a6c7c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7fd3df59e640 1 -- 172.21.15.163:0/3241988472 wait complete. 2024-05-08T06:36:52.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7ffb8159a640 1 -- 172.21.15.163:0/2576389670 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb780016f0 con 0x7ffb840b6160 2024-05-08T06:36:52.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7ffb8159a640 1 -- 172.21.15.163:0/2576389670 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c007540 con 0x7ffb840b6160 2024-05-08T06:36:52.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7ffb8159a640 1 -- 172.21.15.163:0/2576389670 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c007910 con 0x7ffb840b6160 2024-05-08T06:36:52.007 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7f8a017c1640 1 Processor -- start 2024-05-08T06:36:52.007 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7f8e6af7b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50005290 0x7f8e5c0cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.007 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7f8e6af7b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e5c0ccd90 0x7f8e5c034270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.007 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7f8a017c1640 1 -- start start 2024-05-08T06:36:52.008 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7f589af5f640 1 Processor -- start 2024-05-08T06:36:52.008 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7f8e6af7b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e5c0347b0 0x7f8e5c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.008 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7f8e6af7b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c037a80 con 0x7f8e5c0ccd90 2024-05-08T06:36:52.008 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7f8e6af7b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c037900 con 0x7f8e50005290 2024-05-08T06:36:52.008 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7f8e6af7b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8e5c033ee0 con 0x7f8e5c0347b0 2024-05-08T06:36:52.009 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7f324e704640 1 -- 172.21.15.163:0/1715685369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3248166d90 msgr2=0x7f32480b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.009 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7f324e704640 1 --2- 172.21.15.163:0/1715685369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3248166d90 0x7f32480b3f20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3240009e50 tx=0x7f324002a6a0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.009 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7f324e704640 1 -- 172.21.15.163:0/1715685369 shutdown_connections 2024-05-08T06:36:52.009 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7f589af5f640 1 -- start start 2024-05-08T06:36:52.009 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.961+0000 7fce3d616640 1 --2- 172.21.15.163:0/3788688554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce38163ad0 0x7fce38164820 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fce280097a0 tx=0x7fce2802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5470e9e83056f3ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.010 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7f324e704640 1 --2- 172.21.15.163:0/1715685369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32480b4460 0x7f32480d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.010 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7f8e69778640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e5c0ccd90 0x7f8e5c034270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.011 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7f324e704640 1 --2- 172.21.15.163:0/1715685369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3248166d90 0x7f32480b3f20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.013 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7f8e69f79640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50005290 0x7f8e5c0cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.013 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7f8e69f79640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50005290 0x7f8e5c0cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35760/0 (socket says 172.21.15.163:35760) 2024-05-08T06:36:52.013 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7f8e69f79640 1 -- 172.21.15.163:0/2144428769 learned_addr learned my addr 172.21.15.163:0/2144428769 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:52.014 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7ffb88cc8640 1 -- 172.21.15.163:0/2576389670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb840b6160 msgr2=0x7ffb8415ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.014 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7ffb88cc8640 1 --2- 172.21.15.163:0/2576389670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb840b6160 0x7ffb8415ddd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009d90 tx=0x7ffb7c02a5f0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.015 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7f324e704640 1 -- 172.21.15.163:0/1715685369 >> 172.21.15.163:0/1715685369 conn(0x7f324806c740 msgr2=0x7f324806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:52.015 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7f8e6a77a640 1 --2- 172.21.15.163:0/2144428769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e5c0347b0 0x7f8e5c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.016 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7f324e704640 1 -- 172.21.15.163:0/1715685369 shutdown_connections 2024-05-08T06:36:52.016 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7f324e704640 1 -- 172.21.15.163:0/1715685369 wait complete. 2024-05-08T06:36:52.017 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7fa170ada640 1 Processor -- start 2024-05-08T06:36:52.017 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.962+0000 7fa170ada640 1 -- start start 2024-05-08T06:36:52.017 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8e6a77a640 1 -- 172.21.15.163:0/2144428769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50005290 msgr2=0x7f8e5c0cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.018 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8e6a77a640 1 --2- 172.21.15.163:0/2144428769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50005290 0x7f8e5c0cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.018 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8e6a77a640 1 -- 172.21.15.163:0/2144428769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e5c0ccd90 msgr2=0x7f8e5c034270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.018 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8e6a77a640 1 --2- 172.21.15.163:0/2144428769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e5c0ccd90 0x7f8e5c034270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.018 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8e6a77a640 1 -- 172.21.15.163:0/2144428769 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5c0cfae0 con 0x7f8e5c0347b0 2024-05-08T06:36:52.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8a017c1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89e4003a40 0x7f89f40d0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8e69778640 1 --2- 172.21.15.163:0/2144428769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e5c0ccd90 0x7f8e5c034270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:52.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7fd3df59e640 1 Processor -- start 2024-05-08T06:36:52.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8a017c1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f404ff50 0x7f89f4050a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7fd3df59e640 1 -- start start 2024-05-08T06:36:52.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8e6a77a640 1 --2- 172.21.15.163:0/2144428769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e5c0347b0 0x7f8e5c0c8dd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8e64063f70 tx=0x7f8e6406b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f589af5f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5880003dc0 0x7f5894157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8a017c1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f40e2750 0x7f89f4050f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f589af5f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f589415a640 0x7f589415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.024 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8e5a7fc640 1 -- 172.21.15.163:0/2144428769 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64050cb0 con 0x7f8e5c0347b0 2024-05-08T06:36:52.024 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8e5a7fc640 1 -- 172.21.15.163:0/2144428769 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e640b7470 con 0x7f8e5c0347b0 2024-05-08T06:36:52.024 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8e6af7b640 1 -- 172.21.15.163:0/2144428769 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5c0cea40 con 0x7f8e5c0347b0 2024-05-08T06:36:52.024 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8e5a7fc640 1 -- 172.21.15.163:0/2144428769 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e640b7820 con 0x7f8e5c0347b0 2024-05-08T06:36:52.024 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f8e6af7b640 1 -- 172.21.15.163:0/2144428769 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e5c0cf000 con 0x7f8e5c0347b0 2024-05-08T06:36:52.025 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f5899f5d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5880003dc0 0x7f5894157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.025 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f5899f5d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5880003dc0 0x7f5894157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:48354/0 (socket says 172.21.15.163:48354) 2024-05-08T06:36:52.026 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7fa170ada640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154004800 0x7fa16c0af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.026 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7ffb88cc8640 1 -- 172.21.15.163:0/2576389670 shutdown_connections 2024-05-08T06:36:52.026 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7ffb88cc8640 1 --2- 172.21.15.163:0/2576389670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb840b6160 0x7ffb8415ddd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7ffb88cc8640 1 --2- 172.21.15.163:0/2576389670 >> v2:172.21.15.163:3300/0 conn(0x7ffb840d1260 0x7ffb840d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f5899f5d640 1 -- 172.21.15.163:0/2984289568 learned_addr learned my addr 172.21.15.163:0/2984289568 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:52.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7ffb88cc8640 1 -- 172.21.15.163:0/2576389670 >> 172.21.15.163:0/2576389670 conn(0x7ffb8406c740 msgr2=0x7ffb8406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:52.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f589975c640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f589415a640 0x7f589415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f589af5f640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5894158700 0x7f5894158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f324e704640 1 Processor -- start 2024-05-08T06:36:52.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.963+0000 7f89faffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89e4003a40 0x7f89f40d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f89fa7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f404ff50 0x7f89f4050a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa16a59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154004800 0x7fa16c0af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa16a59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154004800 0x7fa16c0af3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44596/0 (socket says 172.21.15.163:44596) 2024-05-08T06:36:52.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa170ada640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa16c14aa50 0x7fa16c14a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f89fa7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f404ff50 0x7f89f4050a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35766/0 (socket says 172.21.15.163:35766) 2024-05-08T06:36:52.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f89fa7fc640 1 -- 172.21.15.163:0/3368431702 learned_addr learned my addr 172.21.15.163:0/3368431702 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:52.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f8a017c1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f89f40383b0 con 0x7f89e4003a40 2024-05-08T06:36:52.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa16a59c640 1 -- 172.21.15.163:0/3415093335 learned_addr learned my addr 172.21.15.163:0/3415093335 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:52.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f8a017c1640 1 -- 172.21.15.163:0/3368431702 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f89f4057bb0 con 0x7f89f404ff50 2024-05-08T06:36:52.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa170ada640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa16c0cee00 0x7fa16c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.034 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f89faffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89e4003a40 0x7f89f40d0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44582/0 (socket says 172.21.15.163:44582) 2024-05-08T06:36:52.034 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f89fb7fe640 1 --2- 172.21.15.163:0/3368431702 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f40e2750 0x7f89f4050f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.034 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f8a017c1640 1 -- 172.21.15.163:0/3368431702 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f89f4038530 con 0x7f89f40e2750 2024-05-08T06:36:52.035 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa170ada640 1 -- 172.21.15.163:0/3415093335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c06f110 con 0x7fa154004800 2024-05-08T06:36:52.035 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa169d9b640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa16c14aa50 0x7fa16c14a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.036 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f589a75e640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5894158700 0x7f5894158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.036 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa170ada640 1 -- 172.21.15.163:0/3415093335 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c06ee30 con 0x7fa16c0cee00 2024-05-08T06:36:52.037 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f589af5f640 1 -- 172.21.15.163:0/2984289568 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f58940b5980 con 0x7f5894158700 2024-05-08T06:36:52.037 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa170ada640 1 -- 172.21.15.163:0/3415093335 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa16c0704b0 con 0x7fa16c14aa50 2024-05-08T06:36:52.038 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f589af5f640 1 -- 172.21.15.163:0/2984289568 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f589415ea70 con 0x7f589415a640 2024-05-08T06:36:52.038 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa16ad9d640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa16c0cee00 0x7fa16c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.039 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f589af5f640 1 -- 172.21.15.163:0/2984289568 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f58940b5b00 con 0x7f5880003dc0 2024-05-08T06:36:52.039 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7ffb88cc8640 1 -- 172.21.15.163:0/2576389670 shutdown_connections 2024-05-08T06:36:52.040 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7ffb88cc8640 1 -- 172.21.15.163:0/2576389670 wait complete. 2024-05-08T06:36:52.040 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f324e704640 1 -- start start 2024-05-08T06:36:52.041 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa16a59c640 1 -- 172.21.15.163:0/3415093335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa16c14aa50 msgr2=0x7fa16c14a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.041 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa16a59c640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa16c14aa50 0x7fa16c14a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.041 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa16a59c640 1 -- 172.21.15.163:0/3415093335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa16c0cee00 msgr2=0x7fa16c1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.041 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa16a59c640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa16c0cee00 0x7fa16c1565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.041 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fa16a59c640 1 -- 172.21.15.163:0/3415093335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16c1f5510 con 0x7fa154004800 2024-05-08T06:36:52.042 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fce3ce15640 1 -- 172.21.15.163:0/3788688554 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28037070 con 0x7fce38163ad0 2024-05-08T06:36:52.042 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fce3ce15640 1 -- 172.21.15.163:0/3788688554 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce28032440 con 0x7fce38163ad0 2024-05-08T06:36:52.042 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7fce3ce15640 1 -- 172.21.15.163:0/3788688554 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28032810 con 0x7fce38163ad0 2024-05-08T06:36:52.043 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7fa16a59c640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154004800 0x7fa16c0af3b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c7c0 tx=0x7fa15c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.044 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f89fa7fc640 1 -- 172.21.15.163:0/3368431702 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f40e2750 msgr2=0x7f89f4050f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.044 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f89fa7fc640 1 --2- 172.21.15.163:0/3368431702 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f40e2750 0x7f89f4050f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.044 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f89fa7fc640 1 -- 172.21.15.163:0/3368431702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89e4003a40 msgr2=0x7f89f40d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.045 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f89fa7fc640 1 --2- 172.21.15.163:0/3368431702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89e4003a40 0x7f89f40d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.045 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.964+0000 7f89fa7fc640 1 -- 172.21.15.163:0/3368431702 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f403a530 con 0x7f89f404ff50 2024-05-08T06:36:52.045 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7f89fa7fc640 1 --2- 172.21.15.163:0/3368431702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f404ff50 0x7f89f4050a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f89f000e370 tx=0x7f89f000e830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.045 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7f89fb7fe640 1 --2- 172.21.15.163:0/3368431702 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f40e2750 0x7f89f4050f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:52.045 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7f89e37fe640 1 -- 172.21.15.163:0/3368431702 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0019070 con 0x7f89f404ff50 2024-05-08T06:36:52.045 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7f89e37fe640 1 -- 172.21.15.163:0/3368431702 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f00052e0 con 0x7f89f404ff50 2024-05-08T06:36:52.045 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7f89e37fe640 1 -- 172.21.15.163:0/3368431702 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0019070 con 0x7f89f404ff50 2024-05-08T06:36:52.045 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7f8a017c1640 1 -- 172.21.15.163:0/3368431702 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f40d08d0 con 0x7f89f404ff50 2024-05-08T06:36:52.045 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7f8a017c1640 1 -- 172.21.15.163:0/3368431702 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f403a7d0 con 0x7f89f404ff50 2024-05-08T06:36:52.046 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7f589a75e640 1 -- 172.21.15.163:0/2984289568 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5880003dc0 msgr2=0x7f5894157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.046 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7f589a75e640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5880003dc0 0x7f5894157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.046 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7f589a75e640 1 -- 172.21.15.163:0/2984289568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f589415a640 msgr2=0x7f589415aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.046 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7f589a75e640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f589415a640 0x7f589415aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.046 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7f589a75e640 1 -- 172.21.15.163:0/2984289568 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58941593f0 con 0x7f5894158700 2024-05-08T06:36:52.046 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7f589a75e640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5894158700 0x7f5894158b80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5894074c00 tx=0x7f588c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.046 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7f588a7fc640 1 -- 172.21.15.163:0/2984289568 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c00dd30 con 0x7f5894158700 2024-05-08T06:36:52.047 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7f588a7fc640 1 -- 172.21.15.163:0/2984289568 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588c018070 con 0x7f5894158700 2024-05-08T06:36:52.047 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7f589af5f640 1 -- 172.21.15.163:0/2984289568 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58941596d0 con 0x7f5894158700 2024-05-08T06:36:52.047 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7fa15affd640 1 -- 172.21.15.163:0/3415093335 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0079d0 con 0x7fa154004800 2024-05-08T06:36:52.047 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7fa15affd640 1 -- 172.21.15.163:0/3415093335 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa15c007b90 con 0x7fa154004800 2024-05-08T06:36:52.047 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7fa15affd640 1 -- 172.21.15.163:0/3415093335 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0113f0 con 0x7fa154004800 2024-05-08T06:36:52.047 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7fa170ada640 1 -- 172.21.15.163:0/3415093335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16c0af8f0 con 0x7fa154004800 2024-05-08T06:36:52.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7fa170ada640 1 -- 172.21.15.163:0/3415093335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa16c1f4c70 con 0x7fa154004800 2024-05-08T06:36:52.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7f8e6af7b640 1 -- 172.21.15.163:0/2144428769 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e5c0cd170 con 0x7f8e5c0347b0 2024-05-08T06:36:52.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7f8e5a7fc640 1 -- 172.21.15.163:0/2144428769 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8e640be060 con 0x7f8e5c0347b0 2024-05-08T06:36:52.049 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7fd3df59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d8006e90 0x7fd3d80e18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.049 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7fd3df59e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3d804ffc0 0x7fd3d80e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.049 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7fd3df59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3d80eacf0 0x7fd3d80e7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.049 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7fd3df59e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d8054930 con 0x7fd3d804ffc0 2024-05-08T06:36:52.049 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7fd3df59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d80547b0 con 0x7fd3d80eacf0 2024-05-08T06:36:52.049 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7fd3df59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d804f540 con 0x7fd3d8006e90 2024-05-08T06:36:52.049 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7fd3de59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d8006e90 0x7fd3d80e18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.049 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.965+0000 7fd3de59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d8006e90 0x7fd3d80e18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:48388/0 (socket says 172.21.15.163:48388) 2024-05-08T06:36:52.049 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7fd3de59c640 1 -- 172.21.15.163:0/4065832692 learned_addr learned my addr 172.21.15.163:0/4065832692 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:52.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7fd3ded9d640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3d80eacf0 0x7fd3d80e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7fd3ddd9b640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3d804ffc0 0x7fd3d80e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7fce4007b640 1 -- 172.21.15.163:0/3788688554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce38163ad0 msgr2=0x7fce38164820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7fce4007b640 1 --2- 172.21.15.163:0/3788688554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce38163ad0 0x7fce38164820 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fce280097a0 tx=0x7fce2802a180 comp rx=0 tx=0).stop 2024-05-08T06:36:52.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7fce4007b640 1 -- 172.21.15.163:0/3788688554 shutdown_connections 2024-05-08T06:36:52.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7fce4007b640 1 --2- 172.21.15.163:0/3788688554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce38163ad0 0x7fce38164820 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7fce4007b640 1 -- 172.21.15.163:0/3788688554 >> 172.21.15.163:0/3788688554 conn(0x7fce3806d300 msgr2=0x7fce3806d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:52.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7ffb88cc8640 1 Processor -- start 2024-05-08T06:36:52.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7ffb88cc8640 1 -- start start 2024-05-08T06:36:52.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7f589af5f640 1 -- 172.21.15.163:0/2984289568 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5894159c90 con 0x7f5894158700 2024-05-08T06:36:52.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7f588a7fc640 1 -- 172.21.15.163:0/2984289568 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c003b90 con 0x7f5894158700 2024-05-08T06:36:52.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7f324e704640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32480b4460 0x7f324814deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7f324e704640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32480d17a0 0x7f324815d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7f324e704640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3248166d90 0x7f324815da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7f324e704640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f32480afc40 con 0x7f32480b4460 2024-05-08T06:36:52.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7f324e704640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f324806f270 con 0x7f32480d17a0 2024-05-08T06:36:52.052 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7f324e704640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f32480afdc0 con 0x7f3248166d90 2024-05-08T06:36:52.052 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7f32477fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32480d17a0 0x7f324815d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.052 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7f32477fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32480d17a0 0x7f324815d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35806/0 (socket says 172.21.15.163:35806) 2024-05-08T06:36:52.052 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.966+0000 7f32477fe640 1 -- 172.21.15.163:0/526291732 learned_addr learned my addr 172.21.15.163:0/526291732 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:52.052 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7f32477fe640 1 -- 172.21.15.163:0/526291732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3248166d90 msgr2=0x7f324815da20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T06:36:52.052 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7f32477fe640 1 --2- 172.21.15.163:0/526291732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3248166d90 0x7f324815da20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.052 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7f32477fe640 1 -- 172.21.15.163:0/526291732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32480b4460 msgr2=0x7f324814deb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T06:36:52.052 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7f32477fe640 1 --2- 172.21.15.163:0/526291732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32480b4460 0x7f324814deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.052 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7f32477fe640 1 -- 172.21.15.163:0/526291732 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324815df60 con 0x7f32480d17a0 2024-05-08T06:36:52.052 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7f8e5a7fc640 1 --2- 172.21.15.163:0/2144428769 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8e48076300 0x7f8e480787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7fd3ded9d640 1 -- 172.21.15.163:0/4065832692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d8006e90 msgr2=0x7fd3d80e18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7f8e5a7fc640 1 -- 172.21.15.163:0/2144428769 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8e64132ac0 con 0x7f8e5c0347b0 2024-05-08T06:36:52.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7fd3ded9d640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d8006e90 0x7fd3d80e18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7fd3ded9d640 1 -- 172.21.15.163:0/4065832692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3d804ffc0 msgr2=0x7fd3d80e6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7fd3ded9d640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3d804ffc0 0x7fd3d80e6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7f8e69f79640 1 --2- 172.21.15.163:0/2144428769 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8e48076300 0x7f8e480787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7fa170ada640 1 -- 172.21.15.163:0/3415093335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa16c06a4b0 con 0x7fa154004800 2024-05-08T06:36:52.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7f8e69f79640 1 --2- 172.21.15.163:0/2144428769 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8e48076300 0x7f8e480787c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8e600075e0 tx=0x7f8e60007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7ffb88cc8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb840b6160 0x7ffb841f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7ffb88cc8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb840d1260 0x7ffb8415b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7ffb88cc8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb8415d3d0 0x7ffb840b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7ffb88cc8640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8406ea90 con 0x7ffb8415d3d0 2024-05-08T06:36:52.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7ffb88cc8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8415d8b0 con 0x7ffb840b6160 2024-05-08T06:36:52.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7ffb88cc8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffb840afb50 con 0x7ffb840d1260 2024-05-08T06:36:52.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7ffb82d9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb8415d3d0 0x7ffb840b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7ffb8259c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb840b6160 0x7ffb841f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7ffb82d9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb8415d3d0 0x7ffb840b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44616/0 (socket says 172.21.15.163:44616) 2024-05-08T06:36:52.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7ffb82d9d640 1 -- 172.21.15.163:0/3555111803 learned_addr learned my addr 172.21.15.163:0/3555111803 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:52.056 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7f8a017c1640 1 -- 172.21.15.163:0/3368431702 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f40053c0 con 0x7f89f404ff50 2024-05-08T06:36:52.056 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.967+0000 7f32477fe640 1 --2- 172.21.15.163:0/526291732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32480d17a0 0x7f324815d4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f323c00b470 tx=0x7f323c00b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.056 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7f3244ff9640 1 -- 172.21.15.163:0/526291732 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c025070 con 0x7f32480d17a0 2024-05-08T06:36:52.056 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7f3244ff9640 1 -- 172.21.15.163:0/526291732 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323c021800 con 0x7f32480d17a0 2024-05-08T06:36:52.056 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7f3244ff9640 1 -- 172.21.15.163:0/526291732 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c021bf0 con 0x7f32480d17a0 2024-05-08T06:36:52.056 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7f324e704640 1 -- 172.21.15.163:0/526291732 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324815e210 con 0x7f32480d17a0 2024-05-08T06:36:52.056 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7f324e704640 1 -- 172.21.15.163:0/526291732 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324815e7d0 con 0x7f32480d17a0 2024-05-08T06:36:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.969+0000 7ffb82d9d640 1 -- 172.21.15.163:0/3555111803 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb840d1260 msgr2=0x7ffb8415b470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T06:36:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.969+0000 7ffb82d9d640 1 --2- 172.21.15.163:0/3555111803 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb840d1260 0x7ffb8415b470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.969+0000 7ffb82d9d640 1 -- 172.21.15.163:0/3555111803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb840b6160 msgr2=0x7ffb841f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.969+0000 7ffb82d9d640 1 --2- 172.21.15.163:0/3555111803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb840b6160 0x7ffb841f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.057 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.970+0000 7f89e37fe640 1 -- 172.21.15.163:0/3368431702 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f89f0007020 con 0x7f89f404ff50 2024-05-08T06:36:52.058 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.969+0000 7f589af5f640 1 -- 172.21.15.163:0/2984289568 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c005290 con 0x7f5894158700 2024-05-08T06:36:52.058 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.969+0000 7f588a7fc640 1 -- 172.21.15.163:0/2984289568 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f588c0050d0 con 0x7f5894158700 2024-05-08T06:36:52.058 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.968+0000 7fd3ded9d640 1 -- 172.21.15.163:0/4065832692 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d80e6540 con 0x7fd3d80eacf0 2024-05-08T06:36:52.058 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.969+0000 7fd3ddd9b640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3d804ffc0 0x7fd3d80e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:52.059 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.969+0000 7fce4007b640 1 -- 172.21.15.163:0/3788688554 shutdown_connections 2024-05-08T06:36:52.059 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.969+0000 7fce4007b640 1 -- 172.21.15.163:0/3788688554 wait complete. 2024-05-08T06:36:52.059 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.969+0000 7ffb82d9d640 1 -- 172.21.15.163:0/3555111803 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb841555a0 con 0x7ffb8415d3d0 2024-05-08T06:36:52.059 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.970+0000 7f324e704640 1 -- 172.21.15.163:0/526291732 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324806a4b0 con 0x7f32480d17a0 2024-05-08T06:36:52.059 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.970+0000 7ffb82d9d640 1 --2- 172.21.15.163:0/3555111803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb8415d3d0 0x7ffb840b26b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009eb0 tx=0x7ffb7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.060 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.970+0000 7f89e37fe640 1 --2- 172.21.15.163:0/3368431702 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f89e80766d0 0x7f89e8078b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.060 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.970+0000 7f89e37fe640 1 -- 172.21.15.163:0/3368431702 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f89f0010040 con 0x7f89f404ff50 2024-05-08T06:36:52.060 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.969+0000 7fa15affd640 1 -- 172.21.15.163:0/3415093335 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa15c011590 con 0x7fa154004800 2024-05-08T06:36:52.060 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.970+0000 7fa15affd640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa14c0763d0 0x7fa14c078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.060 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.970+0000 7fa15affd640 1 -- 172.21.15.163:0/3415093335 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa15c097680 con 0x7fa154004800 2024-05-08T06:36:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.970+0000 7fa169d9b640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa14c0763d0 0x7fa14c078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.970+0000 7f89faffd640 1 --2- 172.21.15.163:0/3368431702 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f89e80766d0 0x7f89e8078b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.970+0000 7fd3ded9d640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3d80eacf0 0x7fd3d80e7520 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3e006ab60 tx=0x7fd3e0069f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.061 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.971+0000 7f588a7fc640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f58780763a0 0x7f5878078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.062 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.971+0000 7fa169d9b640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa14c0763d0 0x7fa14c078890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa160001eb0 tx=0x7fa160033000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.062 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.971+0000 7fce4007b640 1 Processor -- start 2024-05-08T06:36:52.062 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.971+0000 7f588a7fc640 1 -- 172.21.15.163:0/2984289568 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f588c0975c0 con 0x7f5894158700 2024-05-08T06:36:52.062 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.971+0000 7f5899f5d640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f58780763a0 0x7f5878078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.971+0000 7fce4007b640 1 -- start start 2024-05-08T06:36:52.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.971+0000 7f89faffd640 1 --2- 172.21.15.163:0/3368431702 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f89e80766d0 0x7f89e8078b90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f89ec033000 tx=0x7f89ec009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.971+0000 7f5899f5d640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f58780763a0 0x7f5878078860 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5890006fd0 tx=0x7f58900090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.064 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7fce4007b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce380cf8f0 0x7fce380b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.064 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7fce4007b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce38163ad0 0x7fce38162240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.064 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7fce4007b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce38167470 0x7fce38162780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.064 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7fce4007b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fce380d1a50 con 0x7fce38167470 2024-05-08T06:36:52.064 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7fce4007b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fce38164d60 con 0x7fce38163ad0 2024-05-08T06:36:52.064 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7fce4007b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fce380d1bd0 con 0x7fce380cf8f0 2024-05-08T06:36:52.064 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7f8e5a7fc640 1 -- 172.21.15.163:0/2144428769 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8e640fbdb0 con 0x7f8e5c0347b0 2024-05-08T06:36:52.065 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7fce3de17640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce380cf8f0 0x7fce380b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.065 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7fce3e618640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce38167470 0x7fce38162780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.065 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7fce3e618640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce38167470 0x7fce38162780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44632/0 (socket says 172.21.15.163:44632) 2024-05-08T06:36:52.065 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7fce3d616640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce38163ad0 0x7fce38162240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.065 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7fce3de17640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce380cf8f0 0x7fce380b4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:48400/0 (socket says 172.21.15.163:48400) 2024-05-08T06:36:52.065 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7fce3e618640 1 -- 172.21.15.163:0/2689867779 learned_addr learned my addr 172.21.15.163:0/2689867779 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:52.065 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7fce3d616640 1 -- 172.21.15.163:0/2689867779 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce380cf8f0 msgr2=0x7fce380b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.065 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.972+0000 7fce3d616640 1 --2- 172.21.15.163:0/2689867779 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce380cf8f0 0x7fce380b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.066 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.973+0000 7fce3d616640 1 -- 172.21.15.163:0/2689867779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce38167470 msgr2=0x7fce38162780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.066 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.973+0000 7fce3d616640 1 --2- 172.21.15.163:0/2689867779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce38167470 0x7fce38162780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.066 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.973+0000 7fce3d616640 1 -- 172.21.15.163:0/2689867779 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3814e330 con 0x7fce38163ad0 2024-05-08T06:36:52.066 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.973+0000 7fce3e618640 1 --2- 172.21.15.163:0/2689867779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce38167470 0x7fce38162780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:52.066 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.973+0000 7fce3d616640 1 --2- 172.21.15.163:0/2689867779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce38163ad0 0x7fce38162240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fce28031000 tx=0x7fce280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.066 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.974+0000 7f89e37fe640 1 -- 172.21.15.163:0/3368431702 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f89f005f750 con 0x7f89f404ff50 2024-05-08T06:36:52.066 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.974+0000 7fce2e7fc640 1 -- 172.21.15.163:0/2689867779 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28043070 con 0x7fce38163ad0 2024-05-08T06:36:52.067 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.974+0000 7fd3ceffd640 1 -- 172.21.15.163:0/4065832692 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0072e60 con 0x7fd3d80eacf0 2024-05-08T06:36:52.067 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.974+0000 7fce2e7fc640 1 -- 172.21.15.163:0/2689867779 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce28038040 con 0x7fce38163ad0 2024-05-08T06:36:52.067 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.974+0000 7fd3df59e640 1 -- 172.21.15.163:0/4065832692 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d80e6820 con 0x7fd3d80eacf0 2024-05-08T06:36:52.067 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.974+0000 7fd3ceffd640 1 -- 172.21.15.163:0/4065832692 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e00740b0 con 0x7fd3d80eacf0 2024-05-08T06:36:52.067 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.974+0000 7fce4007b640 1 -- 172.21.15.163:0/2689867779 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce381fdd30 con 0x7fce38163ad0 2024-05-08T06:36:52.068 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.974+0000 7fce4007b640 1 -- 172.21.15.163:0/2689867779 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3814e8d0 con 0x7fce38163ad0 2024-05-08T06:36:52.068 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.974+0000 7fce2e7fc640 1 -- 172.21.15.163:0/2689867779 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28043070 con 0x7fce38163ad0 2024-05-08T06:36:52.068 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.975+0000 7fd3ceffd640 1 -- 172.21.15.163:0/4065832692 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0074500 con 0x7fd3d80eacf0 2024-05-08T06:36:52.068 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.975+0000 7fd3df59e640 1 -- 172.21.15.163:0/4065832692 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d804bdd0 con 0x7fd3d80eacf0 2024-05-08T06:36:52.068 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.975+0000 7fce4007b640 1 -- 172.21.15.163:0/2689867779 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce3806a4b0 con 0x7fce38163ad0 2024-05-08T06:36:52.069 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.976+0000 7fa15affd640 1 -- 172.21.15.163:0/3415093335 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa15c015020 con 0x7fa154004800 2024-05-08T06:36:52.069 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.976+0000 7ffb72ffd640 1 -- 172.21.15.163:0/3555111803 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c03f070 con 0x7ffb8415d3d0 2024-05-08T06:36:52.069 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.976+0000 7ffb88cc8640 1 -- 172.21.15.163:0/3555111803 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb84155840 con 0x7ffb8415d3d0 2024-05-08T06:36:52.069 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.976+0000 7ffb88cc8640 1 -- 172.21.15.163:0/3555111803 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb84155e00 con 0x7ffb8415d3d0 2024-05-08T06:36:52.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.975+0000 7f3244ff9640 1 -- 172.21.15.163:0/526291732 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f323c028070 con 0x7f32480d17a0 2024-05-08T06:36:52.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.976+0000 7f3244ff9640 1 --2- 172.21.15.163:0/526291732 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f3234076350 0x7f3234078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.976+0000 7f3244ff9640 1 -- 172.21.15.163:0/526291732 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f323c0a8560 con 0x7f32480d17a0 2024-05-08T06:36:52.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.976+0000 7f3247fff640 1 --2- 172.21.15.163:0/526291732 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f3234076350 0x7f3234078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.977+0000 7fd3df59e640 1 -- 172.21.15.163:0/4065832692 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d8005380 con 0x7fd3d80eacf0 2024-05-08T06:36:52.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.977+0000 7f3247fff640 1 --2- 172.21.15.163:0/526291732 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f3234076350 0x7f3234078810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3240033000 tx=0x7f324002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.977+0000 7ffb88cc8640 1 -- 172.21.15.163:0/3555111803 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb84156000 con 0x7ffb8415d3d0 2024-05-08T06:36:52.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.979+0000 7f3244ff9640 1 -- 172.21.15.163:0/526291732 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f323c071fa0 con 0x7f32480d17a0 2024-05-08T06:36:52.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.981+0000 7fd3ceffd640 1 -- 172.21.15.163:0/4065832692 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd3e00746a0 con 0x7fd3d80eacf0 2024-05-08T06:36:52.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.981+0000 7ffb72ffd640 1 -- 172.21.15.163:0/3555111803 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c007540 con 0x7ffb8415d3d0 2024-05-08T06:36:52.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.981+0000 7ffb72ffd640 1 -- 172.21.15.163:0/3555111803 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c009450 con 0x7ffb8415d3d0 2024-05-08T06:36:52.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.981+0000 7fd3ceffd640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd3c00762d0 0x7fd3c0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.982+0000 7fd3de59c640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd3c00762d0 0x7fd3c0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.982+0000 7fd3ceffd640 1 -- 172.21.15.163:0/4065832692 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd3e0113e80 con 0x7fd3d80eacf0 2024-05-08T06:36:52.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.983+0000 7fd3de59c640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd3c00762d0 0x7fd3c0078790 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0000c00 tx=0x7fd3d002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.983+0000 7f588a7fc640 1 -- 172.21.15.163:0/2984289568 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f588c060f90 con 0x7f5894158700 2024-05-08T06:36:52.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.983+0000 7fce2e7fc640 1 -- 172.21.15.163:0/2689867779 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fce2803c910 con 0x7fce38163ad0 2024-05-08T06:36:52.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.984+0000 7fce2e7fc640 1 --2- 172.21.15.163:0/2689867779 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fce1c0764d0 0x7fce1c078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.984+0000 7fce2e7fc640 1 -- 172.21.15.163:0/2689867779 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fce280b7990 con 0x7fce38163ad0 2024-05-08T06:36:52.075 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.983+0000 7ffb72ffd640 1 -- 172.21.15.163:0/3555111803 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffb7c03b410 con 0x7ffb8415d3d0 2024-05-08T06:36:52.075 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.984+0000 7ffb72ffd640 1 --2- 172.21.15.163:0/3555111803 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffb640762d0 0x7ffb64078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:52.075 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.984+0000 7ffb72ffd640 1 -- 172.21.15.163:0/3555111803 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffb7c0b8e10 con 0x7ffb8415d3d0 2024-05-08T06:36:52.076 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.984+0000 7fd3ceffd640 1 -- 172.21.15.163:0/4065832692 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd3e0090030 con 0x7fd3d80eacf0 2024-05-08T06:36:52.076 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.985+0000 7fce3de17640 1 --2- 172.21.15.163:0/2689867779 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fce1c0764d0 0x7fce1c078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.076 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.985+0000 7ffb8259c640 1 --2- 172.21.15.163:0/3555111803 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffb640762d0 0x7ffb64078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:52.076 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.985+0000 7fce3de17640 1 --2- 172.21.15.163:0/2689867779 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fce1c0764d0 0x7fce1c078990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce34005880 tx=0x7fce34007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.077 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.985+0000 7ffb8259c640 1 --2- 172.21.15.163:0/3555111803 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffb640762d0 0x7ffb64078790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb840cf320 tx=0x7ffb74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:52.077 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.988+0000 7ffb72ffd640 1 -- 172.21.15.163:0/3555111803 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffb84156000 con 0x7ffb8415d3d0 2024-05-08T06:36:52.077 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:51.988+0000 7fce2e7fc640 1 -- 172.21.15.163:0/2689867779 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fce28080c00 con 0x7fce38163ad0 2024-05-08T06:36:52.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.407+0000 7f8e6af7b640 1 -- 172.21.15.163:0/2144428769 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8e5c005380 con 0x7f8e5c0347b0 2024-05-08T06:36:52.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.407+0000 7f8e5a7fc640 1 -- 172.21.15.163:0/2144428769 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8e641000b0 con 0x7f8e5c0347b0 2024-05-08T06:36:52.406 INFO:teuthology.orchestra.run.smithi163.stdout:55834574854 2024-05-08T06:36:52.410 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.411+0000 7f8e47fff640 1 -- 172.21.15.163:0/2144428769 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8e48076300 msgr2=0x7f8e480787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.410 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.411+0000 7f8e47fff640 1 --2- 172.21.15.163:0/2144428769 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8e48076300 0x7f8e480787c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8e600075e0 tx=0x7f8e60007df0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.410 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.411+0000 7f8e47fff640 1 -- 172.21.15.163:0/2144428769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e5c0347b0 msgr2=0x7f8e5c0c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.410 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.411+0000 7f8e47fff640 1 --2- 172.21.15.163:0/2144428769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e5c0347b0 0x7f8e5c0c8dd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8e64063f70 tx=0x7f8e6406b660 comp rx=0 tx=0).stop 2024-05-08T06:36:52.411 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.411+0000 7f8e47fff640 1 -- 172.21.15.163:0/2144428769 shutdown_connections 2024-05-08T06:36:52.411 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.411+0000 7f8e47fff640 1 --2- 172.21.15.163:0/2144428769 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8e48076300 0x7f8e480787c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.411 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.411+0000 7f8e47fff640 1 --2- 172.21.15.163:0/2144428769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e5c0347b0 0x7f8e5c0c8dd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.411 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.411+0000 7f8e47fff640 1 --2- 172.21.15.163:0/2144428769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e5c0ccd90 0x7f8e5c034270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.411 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.411+0000 7f8e47fff640 1 --2- 172.21.15.163:0/2144428769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e50005290 0x7f8e5c0cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.411 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.411+0000 7f8e47fff640 1 -- 172.21.15.163:0/2144428769 >> 172.21.15.163:0/2144428769 conn(0x7f8e5c009fa0 msgr2=0x7f8e5c04c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:52.411 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.412+0000 7f8e47fff640 1 -- 172.21.15.163:0/2144428769 shutdown_connections 2024-05-08T06:36:52.411 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.412+0000 7f8e47fff640 1 -- 172.21.15.163:0/2144428769 wait complete. 2024-05-08T06:36:52.413 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.414+0000 7fa170ada640 1 -- 172.21.15.163:0/3415093335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fa16c0adf00 con 0x7fa154004800 2024-05-08T06:36:52.413 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.414+0000 7f8a017c1640 1 -- 172.21.15.163:0/3368431702 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f89f4004f40 con 0x7f89f404ff50 2024-05-08T06:36:52.413 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.414+0000 7fa15affd640 1 -- 172.21.15.163:0/3415093335 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa15c060a00 con 0x7fa154004800 2024-05-08T06:36:52.414 INFO:teuthology.orchestra.run.smithi163.stdout:55834574854 2024-05-08T06:36:52.414 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.415+0000 7f89e37fe640 1 -- 172.21.15.163:0/3368431702 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f89f0063a50 con 0x7f89f404ff50 2024-05-08T06:36:52.414 INFO:teuthology.orchestra.run.smithi163.stdout:55834574854 2024-05-08T06:36:52.415 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.416+0000 7f589af5f640 1 -- 172.21.15.163:0/2984289568 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f587c005580 con 0x7f5894158700 2024-05-08T06:36:52.415 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.416+0000 7f324e704640 1 -- 172.21.15.163:0/526291732 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f324806cf10 con 0x7f32480d17a0 2024-05-08T06:36:52.416 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.417+0000 7f588a7fc640 1 -- 172.21.15.163:0/2984289568 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f588c060890 con 0x7f5894158700 2024-05-08T06:36:52.416 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2024-05-08T06:36:52.417 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.418+0000 7f3244ff9640 1 -- 172.21.15.163:0/526291732 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f323c0718a0 con 0x7f32480d17a0 2024-05-08T06:36:52.418 INFO:teuthology.orchestra.run.smithi163.stdout:55834574854 2024-05-08T06:36:52.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.418+0000 7fa170ada640 1 -- 172.21.15.163:0/3415093335 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa14c0763d0 msgr2=0x7fa14c078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.418+0000 7fa170ada640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa14c0763d0 0x7fa14c078890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa160001eb0 tx=0x7fa160033000 comp rx=0 tx=0).stop 2024-05-08T06:36:52.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.418+0000 7fa170ada640 1 -- 172.21.15.163:0/3415093335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154004800 msgr2=0x7fa16c0af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.419 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.419+0000 7f8a017c1640 1 -- 172.21.15.163:0/3368431702 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f89e80766d0 msgr2=0x7f89e8078b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.419 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.419+0000 7f8a017c1640 1 --2- 172.21.15.163:0/3368431702 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f89e80766d0 0x7f89e8078b90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f89ec033000 tx=0x7f89ec009030 comp rx=0 tx=0).stop 2024-05-08T06:36:52.419 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.418+0000 7fa170ada640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154004800 0x7fa16c0af3b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c7c0 tx=0x7fa15c00cc80 comp rx=0 tx=0).stop 2024-05-08T06:36:52.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.419+0000 7f8a017c1640 1 -- 172.21.15.163:0/3368431702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f404ff50 msgr2=0x7f89f4050a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.419+0000 7f8a017c1640 1 --2- 172.21.15.163:0/3368431702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f404ff50 0x7f89f4050a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f89f000e370 tx=0x7f89f000e830 comp rx=0 tx=0).stop 2024-05-08T06:36:52.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.419+0000 7f8a017c1640 1 -- 172.21.15.163:0/3368431702 shutdown_connections 2024-05-08T06:36:52.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.419+0000 7f8a017c1640 1 --2- 172.21.15.163:0/3368431702 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f89e80766d0 0x7f89e8078b90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.419+0000 7f8a017c1640 1 --2- 172.21.15.163:0/3368431702 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f40e2750 0x7f89f4050f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.419+0000 7f8a017c1640 1 --2- 172.21.15.163:0/3368431702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f404ff50 0x7f89f4050a10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.419+0000 7f8a017c1640 1 --2- 172.21.15.163:0/3368431702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89e4003a40 0x7f89f40d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.419+0000 7f8a017c1640 1 -- 172.21.15.163:0/3368431702 >> 172.21.15.163:0/3368431702 conn(0x7f89f4055990 msgr2=0x7f89f403c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:52.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.419+0000 7f8a017c1640 1 -- 172.21.15.163:0/3368431702 shutdown_connections 2024-05-08T06:36:52.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.419+0000 7fa170ada640 1 -- 172.21.15.163:0/3415093335 shutdown_connections 2024-05-08T06:36:52.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.419+0000 7f8a017c1640 1 -- 172.21.15.163:0/3368431702 wait complete. 2024-05-08T06:36:52.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.419+0000 7fa170ada640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa14c0763d0 0x7fa14c078890 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.420+0000 7fa170ada640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa16c0cee00 0x7fa16c1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.420+0000 7fa170ada640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa16c14aa50 0x7fa16c14a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.420+0000 7fa170ada640 1 --2- 172.21.15.163:0/3415093335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154004800 0x7fa16c0af3b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.420+0000 7fa170ada640 1 -- 172.21.15.163:0/3415093335 >> 172.21.15.163:0/3415093335 conn(0x7fa16c06c740 msgr2=0x7fa16c072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:52.425 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.420+0000 7fa170ada640 1 -- 172.21.15.163:0/3415093335 shutdown_connections 2024-05-08T06:36:52.425 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.420+0000 7fa170ada640 1 -- 172.21.15.163:0/3415093335 wait complete. 2024-05-08T06:36:52.425 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.420+0000 7f589af5f640 1 -- 172.21.15.163:0/2984289568 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f58780763a0 msgr2=0x7f5878078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.425 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.420+0000 7f589af5f640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f58780763a0 0x7f5878078860 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5890006fd0 tx=0x7f58900090e0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.425 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.420+0000 7f589af5f640 1 -- 172.21.15.163:0/2984289568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5894158700 msgr2=0x7f5894158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.420+0000 7f589af5f640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5894158700 0x7f5894158b80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5894074c00 tx=0x7f588c00d520 comp rx=0 tx=0).stop 2024-05-08T06:36:52.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.421+0000 7f589af5f640 1 -- 172.21.15.163:0/2984289568 shutdown_connections 2024-05-08T06:36:52.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.421+0000 7f589af5f640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f58780763a0 0x7f5878078860 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5890006fd0 tx=0x7f58900090e0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.421+0000 7f589af5f640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5894158700 0x7f5894158b80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.421+0000 7f589af5f640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f589415a640 0x7f589415aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.421+0000 7f589af5f640 1 --2- 172.21.15.163:0/2984289568 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5880003dc0 0x7f5894157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.421+0000 7f589af5f640 1 -- 172.21.15.163:0/2984289568 >> 172.21.15.163:0/2984289568 conn(0x7f58940b29d0 msgr2=0x7f5894153520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:52.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.422+0000 7f589af5f640 1 -- 172.21.15.163:0/2984289568 shutdown_connections 2024-05-08T06:36:52.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.422+0000 7f324e704640 1 -- 172.21.15.163:0/526291732 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f3234076350 msgr2=0x7f3234078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.422+0000 7f324e704640 1 --2- 172.21.15.163:0/526291732 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f3234076350 0x7f3234078810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3240033000 tx=0x7f324002e000 comp rx=0 tx=0).stop 2024-05-08T06:36:52.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.422+0000 7f324e704640 1 -- 172.21.15.163:0/526291732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32480d17a0 msgr2=0x7f324815d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.422+0000 7f324e704640 1 --2- 172.21.15.163:0/526291732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32480d17a0 0x7f324815d4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f323c00b470 tx=0x7f323c00b930 comp rx=0 tx=0).stop 2024-05-08T06:36:52.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.422+0000 7f589af5f640 1 -- 172.21.15.163:0/2984289568 wait complete. 2024-05-08T06:36:52.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.422+0000 7f324e704640 1 -- 172.21.15.163:0/526291732 shutdown_connections 2024-05-08T06:36:52.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.422+0000 7f324e704640 1 --2- 172.21.15.163:0/526291732 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f3234076350 0x7f3234078810 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.422+0000 7f324e704640 1 --2- 172.21.15.163:0/526291732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3248166d90 0x7f324815da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.423+0000 7f324e704640 1 --2- 172.21.15.163:0/526291732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32480d17a0 0x7f324815d4e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.423+0000 7f324e704640 1 --2- 172.21.15.163:0/526291732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32480b4460 0x7f324814deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.423+0000 7f324e704640 1 -- 172.21.15.163:0/526291732 >> 172.21.15.163:0/526291732 conn(0x7f324806c740 msgr2=0x7f3248161760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:52.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.423+0000 7f324e704640 1 -- 172.21.15.163:0/526291732 shutdown_connections 2024-05-08T06:36:52.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.423+0000 7f324e704640 1 -- 172.21.15.163:0/526291732 wait complete. 2024-05-08T06:36:52.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.424+0000 7fd3df59e640 1 -- 172.21.15.163:0/4065832692 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd3d80490d0 con 0x7fd3d80eacf0 2024-05-08T06:36:52.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.425+0000 7fd3ceffd640 1 -- 172.21.15.163:0/4065832692 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd3e00dd200 con 0x7fd3d80eacf0 2024-05-08T06:36:52.429 INFO:teuthology.orchestra.run.smithi163.stdout:55834574856 2024-05-08T06:36:52.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.429+0000 7fd3df59e640 1 -- 172.21.15.163:0/4065832692 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd3c00762d0 msgr2=0x7fd3c0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.429+0000 7fd3df59e640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd3c00762d0 0x7fd3c0078790 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0000c00 tx=0x7fd3d002e000 comp rx=0 tx=0).stop 2024-05-08T06:36:52.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.429+0000 7fd3df59e640 1 -- 172.21.15.163:0/4065832692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3d80eacf0 msgr2=0x7fd3d80e7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.430+0000 7fd3df59e640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3d80eacf0 0x7fd3d80e7520 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3e006ab60 tx=0x7fd3e0069f30 comp rx=0 tx=0).stop 2024-05-08T06:36:52.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.430+0000 7fd3df59e640 1 -- 172.21.15.163:0/4065832692 shutdown_connections 2024-05-08T06:36:52.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.430+0000 7fd3df59e640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd3c00762d0 0x7fd3c0078790 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.430+0000 7fd3df59e640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3d80eacf0 0x7fd3d80e7520 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.430+0000 7fd3df59e640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3d804ffc0 0x7fd3d80e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.430+0000 7fd3df59e640 1 --2- 172.21.15.163:0/4065832692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d8006e90 0x7fd3d80e18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.430+0000 7fd3df59e640 1 -- 172.21.15.163:0/4065832692 >> 172.21.15.163:0/4065832692 conn(0x7fd3d8009fa0 msgr2=0x7fd3d803a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:52.432 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.430+0000 7fd3df59e640 1 -- 172.21.15.163:0/4065832692 shutdown_connections 2024-05-08T06:36:52.432 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.430+0000 7fd3df59e640 1 -- 172.21.15.163:0/4065832692 wait complete. 2024-05-08T06:36:52.432 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-05-08T06:36:52.433 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.432+0000 7ffb88cc8640 1 -- 172.21.15.163:0/3555111803 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ffb8406a4b0 con 0x7ffb8415d3d0 2024-05-08T06:36:52.433 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.432+0000 7fce4007b640 1 -- 172.21.15.163:0/2689867779 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fce380d2e70 con 0x7fce38163ad0 2024-05-08T06:36:52.433 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.433+0000 7fce2e7fc640 1 -- 172.21.15.163:0/2689867779 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fce28089240 con 0x7fce38163ad0 2024-05-08T06:36:52.433 INFO:teuthology.orchestra.run.smithi163.stdout:55834574854 2024-05-08T06:36:52.434 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.433+0000 7ffb72ffd640 1 -- 172.21.15.163:0/3555111803 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffb7c082730 con 0x7ffb8415d3d0 2024-05-08T06:36:52.434 INFO:teuthology.orchestra.run.smithi163.stdout:55834574854 2024-05-08T06:36:52.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7fce4007b640 1 -- 172.21.15.163:0/2689867779 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fce1c0764d0 msgr2=0x7fce1c078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7fce4007b640 1 --2- 172.21.15.163:0/2689867779 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fce1c0764d0 0x7fce1c078990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce34005880 tx=0x7fce34007cf0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7fce4007b640 1 -- 172.21.15.163:0/2689867779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce38163ad0 msgr2=0x7fce38162240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7fce4007b640 1 --2- 172.21.15.163:0/2689867779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce38163ad0 0x7fce38162240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fce28031000 tx=0x7fce280046c0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7fce4007b640 1 -- 172.21.15.163:0/2689867779 shutdown_connections 2024-05-08T06:36:52.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7fce4007b640 1 --2- 172.21.15.163:0/2689867779 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fce1c0764d0 0x7fce1c078990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7fce4007b640 1 --2- 172.21.15.163:0/2689867779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce38167470 0x7fce38162780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7fce4007b640 1 --2- 172.21.15.163:0/2689867779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce38163ad0 0x7fce38162240 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7fce4007b640 1 --2- 172.21.15.163:0/2689867779 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce380cf8f0 0x7fce380b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7ffb88cc8640 1 -- 172.21.15.163:0/3555111803 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffb640762d0 msgr2=0x7ffb64078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7fce4007b640 1 -- 172.21.15.163:0/2689867779 >> 172.21.15.163:0/2689867779 conn(0x7fce3806d300 msgr2=0x7fce380d59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:52.438 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7ffb88cc8640 1 --2- 172.21.15.163:0/3555111803 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffb640762d0 0x7ffb64078790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb840cf320 tx=0x7ffb74008040 comp rx=0 tx=0).stop 2024-05-08T06:36:52.438 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7ffb88cc8640 1 -- 172.21.15.163:0/3555111803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb8415d3d0 msgr2=0x7ffb840b26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:52.438 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7ffb88cc8640 1 --2- 172.21.15.163:0/3555111803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb8415d3d0 0x7ffb840b26b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009eb0 tx=0x7ffb7c004a40 comp rx=0 tx=0).stop 2024-05-08T06:36:52.438 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7fce4007b640 1 -- 172.21.15.163:0/2689867779 shutdown_connections 2024-05-08T06:36:52.439 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7ffb88cc8640 1 -- 172.21.15.163:0/3555111803 shutdown_connections 2024-05-08T06:36:52.439 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7ffb88cc8640 1 --2- 172.21.15.163:0/3555111803 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffb640762d0 0x7ffb64078790 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.439 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7fce4007b640 1 -- 172.21.15.163:0/2689867779 wait complete. 2024-05-08T06:36:52.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7ffb88cc8640 1 --2- 172.21.15.163:0/3555111803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb8415d3d0 0x7ffb840b26b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7ffb88cc8640 1 --2- 172.21.15.163:0/3555111803 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb840d1260 0x7ffb8415b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7ffb88cc8640 1 --2- 172.21.15.163:0/3555111803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb840b6160 0x7ffb841f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:52.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.437+0000 7ffb88cc8640 1 -- 172.21.15.163:0/3555111803 >> 172.21.15.163:0/3555111803 conn(0x7ffb8406c740 msgr2=0x7ffb840d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:52.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.438+0000 7ffb88cc8640 1 -- 172.21.15.163:0/3555111803 shutdown_connections 2024-05-08T06:36:52.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:52.438+0000 7ffb88cc8640 1 -- 172.21.15.163:0/3555111803 wait complete. 2024-05-08T06:36:52.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-05-08T06:36:52.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-05-08T06:36:52.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2024-05-08T06:36:52.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-05-08T06:36:52.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2024-05-08T06:36:52.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-05-08T06:36:52.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-05-08T06:36:53.434 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T06:36:53.444 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T06:36:53.445 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T06:36:53.445 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T06:36:53.447 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T06:36:53.453 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T06:36:53.458 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T06:36:53.458 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T06:36:53.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.600+0000 7fb841e59640 1 Processor -- start 2024-05-08T06:36:53.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.600+0000 7fb841e59640 1 -- start start 2024-05-08T06:36:53.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.602+0000 7fb841e59640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb83c0d0e00 0x7fb83c0740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.602+0000 7fb841e59640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb83c0b6080 0x7fb83c0729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.602+0000 7fb841e59640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb83c06e6b0 con 0x7fb83c16c2b0 2024-05-08T06:36:53.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.602+0000 7fb841e59640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fb83c0b64a0 con 0x7fb83c0d0e00 2024-05-08T06:36:53.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.602+0000 7fb841e59640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb83c08cdf0 con 0x7fb83c0b6080 2024-05-08T06:36:53.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.602+0000 7fb83b7fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb83c0d0e00 0x7fb83c0740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.602+0000 7fb83b7fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb83c0d0e00 0x7fb83c0740c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44634/0 (socket says 172.21.15.163:44634) 2024-05-08T06:36:53.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.602+0000 7fb83b7fe640 1 -- 172.21.15.163:0/2098518401 learned_addr learned my addr 172.21.15.163:0/2098518401 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:53.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.603+0000 7fb83b7fe640 1 -- 172.21.15.163:0/2098518401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb83c0b6080 msgr2=0x7fb83c0729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:53.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.603+0000 7fb83b7fe640 1 --2- 172.21.15.163:0/2098518401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb83c0b6080 0x7fb83c0729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.603+0000 7fb83b7fe640 1 -- 172.21.15.163:0/2098518401 >> v1:172.21.15.192:6789/0 conn(0x7fb83c16c2b0 legacy=0x7fb83c0d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.603+0000 7fb83b7fe640 1 -- 172.21.15.163:0/2098518401 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83c06e990 con 0x7fb83c0d0e00 2024-05-08T06:36:53.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.604+0000 7fb83b7fe640 1 --2- 172.21.15.163:0/2098518401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb83c0d0e00 0x7fb83c0740c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb82c008870 tx=0x7fb82c02c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdff16d4946ebab7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.605+0000 7fb83a7fc640 1 -- 172.21.15.163:0/2098518401 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c032070 con 0x7fb83c0d0e00 2024-05-08T06:36:53.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.605+0000 7fb83a7fc640 1 -- 172.21.15.163:0/2098518401 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb82c036400 con 0x7fb83c0d0e00 2024-05-08T06:36:53.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.606+0000 7fb841e59640 1 -- 172.21.15.163:0/2098518401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb83c0d0e00 msgr2=0x7fb83c0740c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.606+0000 7fb841e59640 1 --2- 172.21.15.163:0/2098518401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb83c0d0e00 0x7fb83c0740c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb82c008870 tx=0x7fb82c02c280 comp rx=0 tx=0).stop 2024-05-08T06:36:53.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.606+0000 7fb841e59640 1 -- 172.21.15.163:0/2098518401 shutdown_connections 2024-05-08T06:36:53.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.606+0000 7fb841e59640 1 --2- 172.21.15.163:0/2098518401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb83c0b6080 0x7fb83c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.606+0000 7fb841e59640 1 --2- 172.21.15.163:0/2098518401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb83c0d0e00 0x7fb83c0740c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.606+0000 7fb841e59640 1 -- 172.21.15.163:0/2098518401 >> 172.21.15.163:0/2098518401 conn(0x7fb83c06c320 msgr2=0x7fb83c06c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:53.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.606+0000 7fb841e59640 1 -- 172.21.15.163:0/2098518401 shutdown_connections 2024-05-08T06:36:53.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.606+0000 7fb841e59640 1 -- 172.21.15.163:0/2098518401 wait complete. 2024-05-08T06:36:53.607 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.608+0000 7fb841e59640 1 Processor -- start 2024-05-08T06:36:53.608 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.608+0000 7fb841e59640 1 -- start start 2024-05-08T06:36:53.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.609+0000 7fb841e59640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb83c0b6080 0x7fb83c165790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.610+0000 7fb841e59640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb83c16c2b0 0x7fb83c08c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.610+0000 7fb83affd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb83c16c2b0 0x7fb83c08c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.610+0000 7fb83affd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb83c16c2b0 0x7fb83c08c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44646/0 (socket says 172.21.15.163:44646) 2024-05-08T06:36:53.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.610+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb83c0b6080 0x7fb83c165790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.610+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb83c0b6080 0x7fb83c165790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35834/0 (socket says 172.21.15.163:35834) 2024-05-08T06:36:53.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.610+0000 7fb83affd640 1 -- 172.21.15.163:0/495153219 learned_addr learned my addr 172.21.15.163:0/495153219 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:53.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.610+0000 7fb841e59640 1 --2- 172.21.15.163:0/495153219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb83c0af5f0 0x7fb83c0afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.610+0000 7fb841e59640 1 -- 172.21.15.163:0/495153219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c06f230 con 0x7fb83c16c2b0 2024-05-08T06:36:53.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.610+0000 7fb841e59640 1 -- 172.21.15.163:0/495153219 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c06ef50 con 0x7fb83c0b6080 2024-05-08T06:36:53.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.610+0000 7fb841e59640 1 -- 172.21.15.163:0/495153219 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb83c06e380 con 0x7fb83c0af5f0 2024-05-08T06:36:53.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.611+0000 7fb83affd640 1 -- 172.21.15.163:0/495153219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb83c0af5f0 msgr2=0x7fb83c0afaa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T06:36:53.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.611+0000 7fb83affd640 1 --2- 172.21.15.163:0/495153219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb83c0af5f0 0x7fb83c0afaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.611+0000 7fb83affd640 1 -- 172.21.15.163:0/495153219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb83c0b6080 msgr2=0x7fb83c165790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.611+0000 7fb83affd640 1 --2- 172.21.15.163:0/495153219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb83c0b6080 0x7fb83c165790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.611+0000 7fb83affd640 1 -- 172.21.15.163:0/495153219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83c205460 con 0x7fb83c16c2b0 2024-05-08T06:36:53.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.611+0000 7fb83affd640 1 --2- 172.21.15.163:0/495153219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb83c16c2b0 0x7fb83c08c2a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb83400caf0 tx=0x7fb83400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.611+0000 7fb823fff640 1 -- 172.21.15.163:0/495153219 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834017070 con 0x7fb83c16c2b0 2024-05-08T06:36:53.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.611+0000 7fb841e59640 1 -- 172.21.15.163:0/495153219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83c205700 con 0x7fb83c16c2b0 2024-05-08T06:36:53.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.611+0000 7fb841e59640 1 -- 172.21.15.163:0/495153219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb83c205cc0 con 0x7fb83c16c2b0 2024-05-08T06:36:53.616 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.615+0000 7fb823fff640 1 -- 172.21.15.163:0/495153219 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb834003b70 con 0x7fb83c16c2b0 2024-05-08T06:36:53.616 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.615+0000 7fb823fff640 1 -- 172.21.15.163:0/495153219 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83400a830 con 0x7fb83c16c2b0 2024-05-08T06:36:53.616 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.615+0000 7fb823fff640 1 -- 172.21.15.163:0/495153219 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb83401a070 con 0x7fb83c16c2b0 2024-05-08T06:36:53.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.617+0000 7fb823fff640 1 --2- 172.21.15.163:0/495153219 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb82407a950 0x7fb82407ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.618+0000 7fb823fff640 1 -- 172.21.15.163:0/495153219 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb834096df0 con 0x7fb83c16c2b0 2024-05-08T06:36:53.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.618+0000 7fb83b7fe640 1 --2- 172.21.15.163:0/495153219 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb82407a950 0x7fb82407ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.618+0000 7fb841e59640 1 -- 172.21.15.163:0/495153219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb828005290 con 0x7fb83c16c2b0 2024-05-08T06:36:53.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.628+0000 7fb823fff640 1 -- 172.21.15.163:0/495153219 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb8340600e0 con 0x7fb83c16c2b0 2024-05-08T06:36:53.628 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.629+0000 7fb83b7fe640 1 --2- 172.21.15.163:0/495153219 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb82407a950 0x7fb82407ce10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb82c037000 tx=0x7fb82c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.656+0000 7f17339af640 1 Processor -- start 2024-05-08T06:36:53.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.657+0000 7f17339af640 1 -- start start 2024-05-08T06:36:53.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.656+0000 7f2b94d12640 1 Processor -- start 2024-05-08T06:36:53.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.657+0000 7f2b94d12640 1 -- start start 2024-05-08T06:36:53.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f5b13740640 1 Processor -- start 2024-05-08T06:36:53.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f2b94d12640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2b90166d90 0x7f2b900b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f2b94d12640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b900b4460 0x7f2b900d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f2b94d12640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b900d17a0 0x7f2b90167330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f2b94d12640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f2b900d1c10 con 0x7f2b900b4460 2024-05-08T06:36:53.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f2b94d12640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2b900adf00 con 0x7f2b90166d90 2024-05-08T06:36:53.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f2b94d12640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2b900b49c0 con 0x7f2b900d17a0 2024-05-08T06:36:53.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7effa997f640 1 Processor -- start 2024-05-08T06:36:53.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f9e0bfff640 1 Processor -- start 2024-05-08T06:36:53.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f5b13740640 1 -- start start 2024-05-08T06:36:53.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7effa997f640 1 -- start start 2024-05-08T06:36:53.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7fa1a3fff640 1 Processor -- start 2024-05-08T06:36:53.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7fa1a3fff640 1 -- start start 2024-05-08T06:36:53.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f5b13740640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b0c149690 0x7f5b0c0727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f5b13740640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b0c149e20 con 0x7f5b0c146790 2024-05-08T06:36:53.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f5b13740640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b0c149b40 con 0x7f5b0c072d20 2024-05-08T06:36:53.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f5b13740640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5b0c0cd910 con 0x7f5b0c149690 2024-05-08T06:36:53.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f9e0bfff640 1 -- start start 2024-05-08T06:36:53.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f9e0bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e0c164760 0x7f9e0c163a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f17339af640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f172c0d0780 0x7f172c16a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f5b114dc640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5b0c146790 0x7f5b0c148f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57140/0 (socket says 172.21.15.163:57140) 2024-05-08T06:36:53.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f2b8dd9b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b900b4460 0x7f2b900d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f2b8e59c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2b90166d90 0x7f2b900b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f2b8e59c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2b90166d90 0x7f2b900b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35858/0 (socket says 172.21.15.163:35858) 2024-05-08T06:36:53.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f2b8dd9b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b900b4460 0x7f2b900d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44660/0 (socket says 172.21.15.163:44660) 2024-05-08T06:36:53.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f9e0bfff640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e0c06c9b0 con 0x7f9e0c0b4cc0 2024-05-08T06:36:53.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f9e0bfff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e0c06cc90 con 0x7f9e0c06da50 2024-05-08T06:36:53.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f9e0bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9e0c06cf70 con 0x7f9e0c164760 2024-05-08T06:36:53.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f2b8e59c640 1 -- 172.21.15.163:0/3004906196 learned_addr learned my addr 172.21.15.163:0/3004906196 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:53.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f2b8ed9d640 1 --2- 172.21.15.163:0/3004906196 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b900d17a0 0x7f2b90167330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7effa997f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effa40b4cb0 0x7effa415b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f2b8e59c640 1 -- 172.21.15.163:0/3004906196 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b900d17a0 msgr2=0x7f2b90167330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:53.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f2b8e59c640 1 --2- 172.21.15.163:0/3004906196 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b900d17a0 0x7f2b90167330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f2b8e59c640 1 -- 172.21.15.163:0/3004906196 >> v2:172.21.15.163:3300/0 conn(0x7f2b900b4460 msgr2=0x7f2b900d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:53.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f2b8e59c640 1 --2- 172.21.15.163:0/3004906196 >> v2:172.21.15.163:3300/0 conn(0x7f2b900b4460 0x7f2b900d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.665 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f2b8e59c640 1 -- 172.21.15.163:0/3004906196 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b900ae150 con 0x7f2b90166d90 2024-05-08T06:36:53.665 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f9e0affd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e0c164760 0x7f9e0c163a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.665 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f5b114dc640 1 -- 172.21.15.163:0/1874746458 learned_addr learned my addr 172.21.15.163:0/1874746458 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:53.665 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f17339af640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f172c0d4f20 con 0x7f172c15f390 2024-05-08T06:36:53.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f9e0affd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e0c164760 0x7f9e0c163a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:48448/0 (socket says 172.21.15.163:48448) 2024-05-08T06:36:53.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f9e0affd640 1 -- 172.21.15.163:0/547811864 learned_addr learned my addr 172.21.15.163:0/547811864 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:53.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f2b8e59c640 1 --2- 172.21.15.163:0/3004906196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b90166d90 0x7f2b900b3f20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b9006f790 tx=0x7f2b8803b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6499b6d6bd004cf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7effa997f640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa40726e0 con 0x7effa40d1aa0 2024-05-08T06:36:53.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7effa997f640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa406e590 con 0x7effa4073ed0 2024-05-08T06:36:53.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7effa997f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7effa40b5090 con 0x7effa40b4cb0 2024-05-08T06:36:53.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.660+0000 7f5b10cdb640 1 --2- 172.21.15.163:0/1874746458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b0c149690 0x7f5b0c0727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f3651d4d640 1 Processor -- start 2024-05-08T06:36:53.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7effa37fe640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7effa40d1aa0 0x7effa415f270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57166/0 (socket says 172.21.15.163:57166) 2024-05-08T06:36:53.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f2b8d59a640 1 -- 172.21.15.163:0/3004906196 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88047070 con 0x7f2b90166d90 2024-05-08T06:36:53.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7effa37fe640 1 -- 172.21.15.163:0/2803496597 learned_addr learned my addr 172.21.15.163:0/2803496597 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:53.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7effa2ffd640 1 --2- 172.21.15.163:0/2803496597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effa40b4cb0 0x7effa415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f17339af640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f172c163f40 con 0x7f172c15f770 2024-05-08T06:36:53.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f17339af640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f172c163980 con 0x7f172c0d0780 2024-05-08T06:36:53.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.658+0000 7f1731f4c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f172c0d0780 0x7f172c16a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f1731f4c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f172c0d0780 0x7f172c16a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35846/0 (socket says 172.21.15.163:35846) 2024-05-08T06:36:53.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f1731f4c640 1 -- 172.21.15.163:0/3796989585 learned_addr learned my addr 172.21.15.163:0/3796989585 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:53.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f1731f4c640 1 -- 172.21.15.163:0/3796989585 >> v1:172.21.15.192:6789/0 conn(0x7f172c15f770 legacy=0x7f172c0d0070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T06:36:53.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f1731f4c640 1 -- 172.21.15.163:0/3796989585 >> v1:172.21.15.163:6789/0 conn(0x7f172c15f390 legacy=0x7f172c163080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f2b8d59a640 1 -- 172.21.15.163:0/3004906196 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b88042600 con 0x7f2b90166d90 2024-05-08T06:36:53.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.659+0000 7f1731f4c640 1 -- 172.21.15.163:0/3796989585 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c163c60 con 0x7f172c0d0780 2024-05-08T06:36:53.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f5b0bfff640 1 -- 172.21.15.163:0/1874746458 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3882218411 0 0) 0x7f5b0c149b40 con 0x7f5b0c072d20 2024-05-08T06:36:53.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f5b0bfff640 1 -- 172.21.15.163:0/1874746458 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5af4003520 con 0x7f5b0c072d20 2024-05-08T06:36:53.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f1731f4c640 1 --2- 172.21.15.163:0/3796989585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c0d0780 0x7f172c16a3b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1724014a00 tx=0x7f1724033280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ccb29954903f217 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7fa1a3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa1a414cfe0 0x7fa1a406f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f5b0bfff640 1 -- 172.21.15.163:0/1874746458 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 446066172 0 0) 0x7f5b0c149e20 con 0x7f5b0c146790 2024-05-08T06:36:53.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7fa1a3fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1a40b5010 0x7fa1a40736d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7effa2ffd640 1 -- 172.21.15.163:0/2803496597 >> v1:172.21.15.192:6789/0 conn(0x7effa4073ed0 legacy=0x7effa40d1390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:36:53.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7effa2ffd640 1 -- 172.21.15.163:0/2803496597 >> v1:172.21.15.163:6789/0 conn(0x7effa40d1aa0 legacy=0x7effa415f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f5b0bfff640 1 -- 172.21.15.163:0/1874746458 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b0c149b40 con 0x7f5b0c146790 2024-05-08T06:36:53.673 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7fa1a3fff640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a4072950 con 0x7fa1a40cd650 2024-05-08T06:36:53.673 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7fa1a3fff640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a414d3c0 con 0x7fa1a40b5010 2024-05-08T06:36:53.673 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7fa1a3fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a4072bf0 con 0x7fa1a414cfe0 2024-05-08T06:36:53.673 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7effa2ffd640 1 -- 172.21.15.163:0/2803496597 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa406e870 con 0x7effa40b4cb0 2024-05-08T06:36:53.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f5b10cdb640 1 -- 172.21.15.163:0/1874746458 >> v1:172.21.15.192:6789/0 conn(0x7f5b0c072d20 legacy=0x7f5b0c06f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f5b10cdb640 1 -- 172.21.15.163:0/1874746458 >> v1:172.21.15.163:6789/0 conn(0x7f5b0c146790 legacy=0x7f5b0c148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f5b0bfff640 1 -- 172.21.15.163:0/1874746458 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2765640379 0 0) 0x7f5b0c149b40 con 0x7f5b0c146790 2024-05-08T06:36:53.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f9e09ffb640 1 -- 172.21.15.163:0/547811864 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1920799765 0 0) 0x7f9e0c06c9b0 con 0x7f9e0c0b4cc0 2024-05-08T06:36:53.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f9e09ffb640 1 -- 172.21.15.163:0/547811864 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9df4003520 con 0x7f9e0c0b4cc0 2024-05-08T06:36:53.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f5b10cdb640 1 -- 172.21.15.163:0/1874746458 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0c06e8a0 con 0x7f5b0c149690 2024-05-08T06:36:53.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f9e09ffb640 1 -- 172.21.15.163:0/547811864 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3994190439 0 0) 0x7f9df4003520 con 0x7f9e0c0b4cc0 2024-05-08T06:36:53.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f9e09ffb640 1 -- 172.21.15.163:0/547811864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e0c164760 msgr2=0x7f9e0c163a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:53.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f9e09ffb640 1 --2- 172.21.15.163:0/547811864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e0c164760 0x7f9e0c163a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f9e09ffb640 1 -- 172.21.15.163:0/547811864 >> v1:172.21.15.192:6789/0 conn(0x7f9e0c06da50 legacy=0x7f9e0c0b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:53.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f9e09ffb640 1 -- 172.21.15.163:0/547811864 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0c06d250 con 0x7f9e0c0b4cc0 2024-05-08T06:36:53.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f9e09ffb640 1 -- 172.21.15.163:0/547811864 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f9e000016d0 con 0x7f9e0c0b4cc0 2024-05-08T06:36:53.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f9e09ffb640 1 -- 172.21.15.163:0/547811864 >> v1:172.21.15.163:6789/0 conn(0x7f9e0c0b4cc0 legacy=0x7f9e0c06e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f9e09ffb640 1 --2- 172.21.15.163:0/547811864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9df4003a20 0x7f9df4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f9e09ffb640 1 --2- 172.21.15.163:0/547811864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df4004800 0x7f9df4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f9e09ffb640 1 --2- 172.21.15.163:0/547811864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9df40051c0 0x7f9df400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f9e09ffb640 1 -- 172.21.15.163:0/547811864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c06cf70 con 0x7f9df4004800 2024-05-08T06:36:53.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f9e09ffb640 1 -- 172.21.15.163:0/547811864 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9df400c060 con 0x7f9df4003a20 2024-05-08T06:36:53.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f9e09ffb640 1 -- 172.21.15.163:0/547811864 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9df400c260 con 0x7f9df40051c0 2024-05-08T06:36:53.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f2b94d12640 1 -- 172.21.15.163:0/3004906196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b90166d90 msgr2=0x7f2b900b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f2b94d12640 1 --2- 172.21.15.163:0/3004906196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b90166d90 0x7f2b900b3f20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b9006f790 tx=0x7f2b8803b6b0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.661+0000 7f3651d4d640 1 -- start start 2024-05-08T06:36:53.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f1723fff640 1 -- 172.21.15.163:0/3796989585 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17240110d0 con 0x7f172c0d0780 2024-05-08T06:36:53.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f9e0affd640 1 --2- 172.21.15.163:0/547811864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9df4003a20 0x7f9df4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f9e0b7fe640 1 --2- 172.21.15.163:0/547811864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9df40051c0 0x7f9df400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f1723fff640 1 -- 172.21.15.163:0/3796989585 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1724011270 con 0x7f172c0d0780 2024-05-08T06:36:53.678 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f9e0a7fc640 1 --2- 172.21.15.163:0/547811864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df4004800 0x7f9df4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.678 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f3651d4d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f364c0d0710 0x7f364c15b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.678 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f1723fff640 1 -- 172.21.15.163:0/3796989585 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724011680 con 0x7f172c0d0780 2024-05-08T06:36:53.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f9e0b7fe640 1 -- 172.21.15.163:0/547811864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9df4003a20 msgr2=0x7f9df4003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:53.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f3651d4d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f364c073750 con 0x7f364c0d1710 2024-05-08T06:36:53.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f9e0b7fe640 1 --2- 172.21.15.163:0/547811864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9df4003a20 0x7f9df4003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f3650d4b640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f364c0d1710 0x7f364c0b3cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57176/0 (socket says 172.21.15.163:57176) 2024-05-08T06:36:53.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f3650d4b640 1 -- 172.21.15.163:0/2883484910 learned_addr learned my addr 172.21.15.163:0/2883484910 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:53.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f3651d4d640 1 -- 172.21.15.163:0/2883484910 --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f364c073a30 con 0x7f364c0b4c30 2024-05-08T06:36:53.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f3651d4d640 1 -- 172.21.15.163:0/2883484910 --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f364c06e500 con 0x7f364c0d0710 2024-05-08T06:36:53.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f9e0b7fe640 1 -- 172.21.15.163:0/547811864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df4004800 msgr2=0x7f9df4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:53.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7effa2ffd640 1 --2- 172.21.15.163:0/2803496597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effa40b4cb0 0x7effa415b950 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0097d0 tx=0x7eff8c02a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=294444955cd50158 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7f9e0b7fe640 1 --2- 172.21.15.163:0/547811864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df4004800 0x7f9df4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f9e0b7fe640 1 -- 172.21.15.163:0/547811864 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0c06d250 con 0x7f9df40051c0 2024-05-08T06:36:53.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 605325960 0 0) 0x7f364c073750 con 0x7f364c0d1710 2024-05-08T06:36:53.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f363c005730 con 0x7f364c0d1710 2024-05-08T06:36:53.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f9e0a7fc640 1 --2- 172.21.15.163:0/547811864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df4004800 0x7f9df4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:53.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f2b94d12640 1 -- 172.21.15.163:0/3004906196 shutdown_connections 2024-05-08T06:36:53.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f2b94d12640 1 --2- 172.21.15.163:0/3004906196 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b900d17a0 0x7f2b90167330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f2b94d12640 1 --2- 172.21.15.163:0/3004906196 >> v2:172.21.15.163:3300/0 conn(0x7f2b900b4460 0x7f2b900d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f2b94d12640 1 --2- 172.21.15.163:0/3004906196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b90166d90 0x7f2b900b3f20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f2b94d12640 1 -- 172.21.15.163:0/3004906196 >> 172.21.15.163:0/3004906196 conn(0x7f2b9006c740 msgr2=0x7f2b9006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:53.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364bfff640 1 --2- 172.21.15.163:0/2883484910 >> v2:172.21.15.192:3300/0 conn(0x7f364c0d0710 0x7f364c15b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7fa1a2ffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa1a414cfe0 0x7fa1a406f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7fa1a2ffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa1a414cfe0 0x7fa1a406f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35882/0 (socket says 172.21.15.163:35882) 2024-05-08T06:36:53.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.662+0000 7fa1a2ffd640 1 -- 172.21.15.163:0/1316942997 learned_addr learned my addr 172.21.15.163:0/1316942997 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:53.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7effa1ffb640 1 -- 172.21.15.163:0/2803496597 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c037070 con 0x7effa40b4cb0 2024-05-08T06:36:53.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7effa1ffb640 1 -- 172.21.15.163:0/2803496597 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff8c032440 con 0x7effa40b4cb0 2024-05-08T06:36:53.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f17339af640 1 -- 172.21.15.163:0/3796989585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c0d0780 msgr2=0x7f172c16a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f17339af640 1 --2- 172.21.15.163:0/3796989585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c0d0780 0x7f172c16a3b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1724014a00 tx=0x7f1724033280 comp rx=0 tx=0).stop 2024-05-08T06:36:53.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f5b10cdb640 1 --2- 172.21.15.163:0/1874746458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b0c149690 0x7f5b0c0727e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009df0 tx=0x7f5afc02a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84cfed484a88c296 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3312016805 0 0) 0x7f364c073a30 con 0x7f364c0b4c30 2024-05-08T06:36:53.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f364c073750 con 0x7f364c0b4c30 2024-05-08T06:36:53.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2041687444 0 0) 0x7f363c005730 con 0x7f364c0d1710 2024-05-08T06:36:53.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 >> v2:172.21.15.192:3300/0 conn(0x7f364c0d0710 msgr2=0x7f364c15b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:53.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 --2- 172.21.15.163:0/2883484910 >> v2:172.21.15.192:3300/0 conn(0x7f364c0d0710 0x7f364c15b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 >> v1:172.21.15.192:6789/0 conn(0x7f364c0b4c30 legacy=0x7f364c165e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364c073d10 con 0x7f364c0d1710 2024-05-08T06:36:53.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7fa1a2ffd640 1 -- 172.21.15.163:0/1316942997 >> v2:172.21.15.163:3300/0 conn(0x7fa1a40b5010 msgr2=0x7fa1a40736d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:53.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7fa1a2ffd640 1 --2- 172.21.15.163:0/1316942997 >> v2:172.21.15.163:3300/0 conn(0x7fa1a40b5010 0x7fa1a40736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7fa1a2ffd640 1 -- 172.21.15.163:0/1316942997 >> v1:172.21.15.163:6789/0 conn(0x7fa1a40cd650 legacy=0x7fa1a40b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:53.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7effa1ffb640 1 -- 172.21.15.163:0/2803496597 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c032810 con 0x7effa40b4cb0 2024-05-08T06:36:53.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f36400016f0 con 0x7f364c0d1710 2024-05-08T06:36:53.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 >> v1:172.21.15.163:6789/0 conn(0x7f364c0d1710 legacy=0x7f364c0b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 --2- 172.21.15.163:0/2883484910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c005e40 0x7f363c006240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f17339af640 1 -- 172.21.15.163:0/3796989585 shutdown_connections 2024-05-08T06:36:53.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f17339af640 1 --2- 172.21.15.163:0/3796989585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c0d0780 0x7f172c16a3b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f17339af640 1 -- 172.21.15.163:0/3796989585 >> 172.21.15.163:0/3796989585 conn(0x7f172c06d300 msgr2=0x7f172c06d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:53.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f5b0bfff640 1 -- 172.21.15.163:0/1874746458 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b000016d0 con 0x7f5b0c149690 2024-05-08T06:36:53.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f5b0bfff640 1 -- 172.21.15.163:0/1874746458 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5afc007540 con 0x7f5b0c149690 2024-05-08T06:36:53.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f5b0bfff640 1 -- 172.21.15.163:0/1874746458 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc007710 con 0x7f5b0c149690 2024-05-08T06:36:53.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 --2- 172.21.15.163:0/2883484910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f363c006a10 0x7f363c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f3650d4b640 1 --2- 172.21.15.163:0/2883484910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c005e40 0x7f363c006240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 --2- 172.21.15.163:0/2883484910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f363c0073d0 0x7f363c0147c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f364c06e500 con 0x7f363c005e40 2024-05-08T06:36:53.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f363c00bc20 con 0x7f363c006a10 2024-05-08T06:36:53.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f363c00be20 con 0x7f363c0073d0 2024-05-08T06:36:53.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f2b94d12640 1 -- 172.21.15.163:0/3004906196 shutdown_connections 2024-05-08T06:36:53.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7f2b94d12640 1 -- 172.21.15.163:0/3004906196 wait complete. 2024-05-08T06:36:53.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.663+0000 7fa1a2ffd640 1 -- 172.21.15.163:0/1316942997 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a4072ed0 con 0x7fa1a414cfe0 2024-05-08T06:36:53.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7effa997f640 1 -- 172.21.15.163:0/2803496597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effa40b4cb0 msgr2=0x7effa415b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7f5b13740640 1 -- 172.21.15.163:0/1874746458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b0c149690 msgr2=0x7f5b0c0727e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7effa997f640 1 --2- 172.21.15.163:0/2803496597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effa40b4cb0 0x7effa415b950 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0097d0 tx=0x7eff8c02a160 comp rx=0 tx=0).stop 2024-05-08T06:36:53.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7f5b13740640 1 --2- 172.21.15.163:0/1874746458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b0c149690 0x7f5b0c0727e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009df0 tx=0x7f5afc02a6a0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7f17339af640 1 -- 172.21.15.163:0/3796989585 shutdown_connections 2024-05-08T06:36:53.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7effa997f640 1 -- 172.21.15.163:0/2803496597 shutdown_connections 2024-05-08T06:36:53.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7f5b13740640 1 -- 172.21.15.163:0/1874746458 shutdown_connections 2024-05-08T06:36:53.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7f5b13740640 1 --2- 172.21.15.163:0/1874746458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b0c149690 0x7f5b0c0727e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7f5b13740640 1 -- 172.21.15.163:0/1874746458 >> 172.21.15.163:0/1874746458 conn(0x7f5b0c0682f0 msgr2=0x7f5b0c06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:53.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7effa997f640 1 --2- 172.21.15.163:0/2803496597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effa40b4cb0 0x7effa415b950 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7effa997f640 1 -- 172.21.15.163:0/2803496597 >> 172.21.15.163:0/2803496597 conn(0x7effa406c740 msgr2=0x7effa406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:53.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7f3650d4b640 1 -- 172.21.15.163:0/2883484910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f363c0073d0 msgr2=0x7f363c0147c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:53.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7f3650d4b640 1 --2- 172.21.15.163:0/2883484910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f363c0073d0 0x7f363c0147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7f3650d4b640 1 -- 172.21.15.163:0/2883484910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f363c006a10 msgr2=0x7f363c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:53.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7f3650d4b640 1 --2- 172.21.15.163:0/2883484910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f363c006a10 0x7f363c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7f3650d4b640 1 -- 172.21.15.163:0/2883484910 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c005730 con 0x7f363c005e40 2024-05-08T06:36:53.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7f365154c640 1 -- 172.21.15.163:0/2883484910 reap_dead start 2024-05-08T06:36:53.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7f5b13740640 1 -- 172.21.15.163:0/1874746458 shutdown_connections 2024-05-08T06:36:53.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7f9e0b7fe640 1 --2- 172.21.15.163:0/547811864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9df40051c0 0x7f9df400b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c0b3260 tx=0x7f9df4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=340a1b1db6cce95a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.693 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.664+0000 7f17339af640 1 -- 172.21.15.163:0/3796989585 wait complete. 2024-05-08T06:36:53.693 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7effa997f640 1 -- 172.21.15.163:0/2803496597 shutdown_connections 2024-05-08T06:36:53.693 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7effa997f640 1 -- 172.21.15.163:0/2803496597 wait complete. 2024-05-08T06:36:53.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7f9e09ffb640 1 -- 172.21.15.163:0/547811864 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00036070 con 0x7f9df40051c0 2024-05-08T06:36:53.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7f9e09ffb640 1 -- 172.21.15.163:0/547811864 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e00007250 con 0x7f9df40051c0 2024-05-08T06:36:53.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7f9e09ffb640 1 -- 172.21.15.163:0/547811864 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00036070 con 0x7f9df40051c0 2024-05-08T06:36:53.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7f3650d4b640 1 --2- 172.21.15.163:0/2883484910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c005e40 0x7f363c006240 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f364000c160 tx=0x7f363c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726c0d845e28736a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7fa1a2ffd640 1 --2- 172.21.15.163:0/1316942997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a414cfe0 0x7fa1a406f2c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40d17d0 tx=0x7fa19402a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46dee2ec0a8dab83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7f2b94d12640 1 Processor -- start 2024-05-08T06:36:53.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7f2b94d12640 1 -- start start 2024-05-08T06:36:53.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640007cc0 con 0x7f363c005e40 2024-05-08T06:36:53.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3640005ad0 con 0x7f363c005e40 2024-05-08T06:36:53.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7f364b7fe640 1 -- 172.21.15.163:0/2883484910 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640007070 con 0x7f363c005e40 2024-05-08T06:36:53.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.666+0000 7f9e0bfff640 1 -- 172.21.15.163:0/547811864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9df40051c0 msgr2=0x7f9df400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.666+0000 7f9e0bfff640 1 --2- 172.21.15.163:0/547811864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9df40051c0 0x7f9df400b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c0b3260 tx=0x7f9df4005830 comp rx=0 tx=0).stop 2024-05-08T06:36:53.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.666+0000 7f9e0b7fe640 1 -- 172.21.15.163:0/547811864 reap_dead start 2024-05-08T06:36:53.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7fa1a1ffb640 1 -- 172.21.15.163:0/1316942997 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194037070 con 0x7fa1a414cfe0 2024-05-08T06:36:53.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7fa1a1ffb640 1 -- 172.21.15.163:0/1316942997 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa194032440 con 0x7fa1a414cfe0 2024-05-08T06:36:53.697 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.666+0000 7f9e0bfff640 1 -- 172.21.15.163:0/547811864 shutdown_connections 2024-05-08T06:36:53.697 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.666+0000 7f9e0bfff640 1 -- 172.21.15.163:0/547811864 >> 172.21.15.163:0/547811864 conn(0x7f9e0c0b32b0 msgr2=0x7f9e0c0b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:53.697 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7fa1a1ffb640 1 -- 172.21.15.163:0/1316942997 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194037070 con 0x7fa1a414cfe0 2024-05-08T06:36:53.697 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.666+0000 7f9e0bfff640 1 -- 172.21.15.163:0/547811864 shutdown_connections 2024-05-08T06:36:53.698 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7f5b13740640 1 -- 172.21.15.163:0/1874746458 wait complete. 2024-05-08T06:36:53.698 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.666+0000 7f2b94d12640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b900b4460 0x7f2b9015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.698 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.666+0000 7f5b13740640 1 Processor -- start 2024-05-08T06:36:53.698 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f5b13740640 1 -- start start 2024-05-08T06:36:53.699 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.666+0000 7f2b94d12640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b900d17a0 0x7f2b9015a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.699 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f2b94d12640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b90166d90 0x7f2b900b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.699 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f2b94d12640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9006f270 con 0x7f2b900b4460 2024-05-08T06:36:53.699 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f2b94d12640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2b900ae710 con 0x7f2b90166d90 2024-05-08T06:36:53.699 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f2b94d12640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2b90167870 con 0x7f2b900d17a0 2024-05-08T06:36:53.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7fa1a3fff640 1 -- 172.21.15.163:0/1316942997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a414cfe0 msgr2=0x7fa1a406f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7fa1a3fff640 1 --2- 172.21.15.163:0/1316942997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a414cfe0 0x7fa1a406f2c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40d17d0 tx=0x7fa19402a290 comp rx=0 tx=0).stop 2024-05-08T06:36:53.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f2b8dd9b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b900d17a0 0x7f2b9015a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7f17339af640 1 Processor -- start 2024-05-08T06:36:53.701 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7effa997f640 1 Processor -- start 2024-05-08T06:36:53.701 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f2b8ed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b90166d90 0x7f2b900b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.701 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f2b8ed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b90166d90 0x7f2b900b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35898/0 (socket says 172.21.15.163:35898) 2024-05-08T06:36:53.701 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f2b8ed9d640 1 -- 172.21.15.163:0/603239468 learned_addr learned my addr 172.21.15.163:0/603239468 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:53.701 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.666+0000 7f9e0bfff640 1 -- 172.21.15.163:0/547811864 wait complete. 2024-05-08T06:36:53.702 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f3651d4d640 1 -- 172.21.15.163:0/2883484910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c005e40 msgr2=0x7f363c006240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.702 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.665+0000 7f17339af640 1 -- start start 2024-05-08T06:36:53.702 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f2b8e59c640 1 --2- 172.21.15.163:0/603239468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b900b4460 0x7f2b9015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f17339af640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f172c0d0780 0x7f172c0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f17339af640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f172c15f390 0x7f172c159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f17339af640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c15f770 0x7f172c0d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f17339af640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f172c0b83d0 con 0x7f172c15f390 2024-05-08T06:36:53.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f17339af640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f172c0d39f0 con 0x7f172c15f770 2024-05-08T06:36:53.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f17339af640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f172c0b8550 con 0x7f172c0d0780 2024-05-08T06:36:53.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f1731f4c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c15f770 0x7f172c0d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f1731f4c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c15f770 0x7f172c0d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35910/0 (socket says 172.21.15.163:35910) 2024-05-08T06:36:53.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f1731f4c640 1 -- 172.21.15.163:0/1027129708 learned_addr learned my addr 172.21.15.163:0/1027129708 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:53.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f173174b640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f172c0d0780 0x7f172c0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f1730f4a640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f172c15f390 0x7f172c159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f3651d4d640 1 --2- 172.21.15.163:0/2883484910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c005e40 0x7f363c006240 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f364000c160 tx=0x7f363c007a40 comp rx=0 tx=0).stop 2024-05-08T06:36:53.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f3651d4d640 1 -- 172.21.15.163:0/2883484910 shutdown_connections 2024-05-08T06:36:53.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f3651d4d640 1 --2- 172.21.15.163:0/2883484910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c005e40 0x7f363c006240 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f364000c160 tx=0x7f363c007a40 comp rx=0 tx=0).stop 2024-05-08T06:36:53.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f3651d4d640 1 -- 172.21.15.163:0/2883484910 >> 172.21.15.163:0/2883484910 conn(0x7f364c0d0c20 msgr2=0x7f364c0d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:53.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7effa997f640 1 -- start start 2024-05-08T06:36:53.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7fa1a3fff640 1 -- 172.21.15.163:0/1316942997 shutdown_connections 2024-05-08T06:36:53.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7fa1a3fff640 1 --2- 172.21.15.163:0/1316942997 >> v2:172.21.15.163:3300/0 conn(0x7fa1a40b5010 0x7fa1a40736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7fa1a3fff640 1 --2- 172.21.15.163:0/1316942997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a414cfe0 0x7fa1a406f2c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7fa1a3fff640 1 -- 172.21.15.163:0/1316942997 >> 172.21.15.163:0/1316942997 conn(0x7fa1a40d0c20 msgr2=0x7fa1a40d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:53.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f2b8ed9d640 1 -- 172.21.15.163:0/603239468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b900d17a0 msgr2=0x7f2b9015a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f1731f4c640 1 -- 172.21.15.163:0/1027129708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f172c0d0780 msgr2=0x7f172c0b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f1731f4c640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f172c0d0780 0x7f172c0b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f1731f4c640 1 -- 172.21.15.163:0/1027129708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f172c15f390 msgr2=0x7f172c159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f1731f4c640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f172c15f390 0x7f172c159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f1731f4c640 1 -- 172.21.15.163:0/1027129708 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c0b4d00 con 0x7f172c15f770 2024-05-08T06:36:53.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f1730f4a640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f172c15f390 0x7f172c159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:53.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f2b8ed9d640 1 --2- 172.21.15.163:0/603239468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b900d17a0 0x7f2b9015a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f2b8ed9d640 1 -- 172.21.15.163:0/603239468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b900b4460 msgr2=0x7f2b9015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f2b8ed9d640 1 --2- 172.21.15.163:0/603239468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b900b4460 0x7f2b9015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.667+0000 7f2b8ed9d640 1 -- 172.21.15.163:0/603239468 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9015adf0 con 0x7f2b90166d90 2024-05-08T06:36:53.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f2b8ed9d640 1 --2- 172.21.15.163:0/603239468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b90166d90 0x7f2b900b1710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000e830 tx=0x7f2b8000ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7fa1a3fff640 1 -- 172.21.15.163:0/1316942997 shutdown_connections 2024-05-08T06:36:53.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7fa1a3fff640 1 -- 172.21.15.163:0/1316942997 wait complete. 2024-05-08T06:36:53.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f3651d4d640 1 -- 172.21.15.163:0/2883484910 shutdown_connections 2024-05-08T06:36:53.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f2b7effd640 1 -- 172.21.15.163:0/603239468 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80017070 con 0x7f2b90166d90 2024-05-08T06:36:53.708 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f1731f4c640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c15f770 0x7f172c0d1cb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1724014a00 tx=0x7f1724004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.708 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f5b13740640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b0c072d20 0x7f5b0c161240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.708 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f5b13740640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b0c146790 0x7f5b0c161780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.708 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f5b13740640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b0c149690 0x7f5b0c0cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.708 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f5b13740640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c146b70 con 0x7f5b0c072d20 2024-05-08T06:36:53.708 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f5b13740640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c06eef0 con 0x7f5b0c149690 2024-05-08T06:36:53.708 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f5b13740640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5b0c06fa90 con 0x7f5b0c146790 2024-05-08T06:36:53.709 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f2b94d12640 1 -- 172.21.15.163:0/603239468 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9015b0d0 con 0x7f2b90166d90 2024-05-08T06:36:53.709 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f2b94d12640 1 -- 172.21.15.163:0/603239468 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b9015b690 con 0x7f2b90166d90 2024-05-08T06:36:53.709 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f2b7effd640 1 -- 172.21.15.163:0/603239468 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b8000c940 con 0x7f2b90166d90 2024-05-08T06:36:53.709 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f2b7effd640 1 -- 172.21.15.163:0/603239468 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8000cd10 con 0x7f2b90166d90 2024-05-08T06:36:53.709 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f5b114dc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b0c072d20 0x7f5b0c161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.709 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7f5b11cdd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b0c149690 0x7f5b0c0cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.709 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7f5b114dc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b0c072d20 0x7f5b0c161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44726/0 (socket says 172.21.15.163:44726) 2024-05-08T06:36:53.709 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7f5b114dc640 1 -- 172.21.15.163:0/3323313998 learned_addr learned my addr 172.21.15.163:0/3323313998 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:53.710 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.668+0000 7f3651d4d640 1 -- 172.21.15.163:0/2883484910 wait complete. 2024-05-08T06:36:53.710 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7f9e0bfff640 1 Processor -- start 2024-05-08T06:36:53.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7effa997f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4073ed0 0x7effa4158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7effa997f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effa40b4cb0 0x7effa41e5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7effa997f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effa40d1aa0 0x7effa41e6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7effa2ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4073ed0 0x7effa4158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7f5b114dc640 1 -- 172.21.15.163:0/3323313998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b0c146790 msgr2=0x7f5b0c161780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T06:36:53.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7f5b114dc640 1 --2- 172.21.15.163:0/3323313998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b0c146790 0x7f5b0c161780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7f5b114dc640 1 -- 172.21.15.163:0/3323313998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b0c149690 msgr2=0x7f5b0c0cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7f5b114dc640 1 --2- 172.21.15.163:0/3323313998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b0c149690 0x7f5b0c0cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7f5b114dc640 1 -- 172.21.15.163:0/3323313998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0c1fa2f0 con 0x7f5b0c072d20 2024-05-08T06:36:53.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7effa2ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4073ed0 0x7effa4158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44734/0 (socket says 172.21.15.163:44734) 2024-05-08T06:36:53.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7effa2ffd640 1 -- 172.21.15.163:0/4181133707 learned_addr learned my addr 172.21.15.163:0/4181133707 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:53.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7effa27fc640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effa40b4cb0 0x7effa41e5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f5b114dc640 1 --2- 172.21.15.163:0/3323313998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b0c072d20 0x7f5b0c161240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5b00007fd0 tx=0x7f5b0000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7effa37fe640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effa40d1aa0 0x7effa41e6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7effa997f640 1 -- 172.21.15.163:0/4181133707 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7effa415be90 con 0x7effa4073ed0 2024-05-08T06:36:53.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7f1721ffb640 1 -- 172.21.15.163:0/1027129708 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172403bbb0 con 0x7f172c15f770 2024-05-08T06:36:53.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7effa997f640 1 -- 172.21.15.163:0/4181133707 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7effa406ee30 con 0x7effa40d1aa0 2024-05-08T06:36:53.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7effa997f640 1 -- 172.21.15.163:0/4181133707 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7effa406f110 con 0x7effa40b4cb0 2024-05-08T06:36:53.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f5b09ffb640 1 -- 172.21.15.163:0/3323313998 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00017070 con 0x7f5b0c072d20 2024-05-08T06:36:53.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7f1721ffb640 1 -- 172.21.15.163:0/1027129708 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172403bd50 con 0x7f172c15f770 2024-05-08T06:36:53.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f1721ffb640 1 -- 172.21.15.163:0/1027129708 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172400f290 con 0x7f172c15f770 2024-05-08T06:36:53.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f17339af640 1 -- 172.21.15.163:0/1027129708 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172c1491e0 con 0x7f172c15f770 2024-05-08T06:36:53.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f17339af640 1 -- 172.21.15.163:0/1027129708 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172c0b5280 con 0x7f172c15f770 2024-05-08T06:36:53.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f17339af640 1 -- 172.21.15.163:0/1027129708 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f172c06a4b0 con 0x7f172c15f770 2024-05-08T06:36:53.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f5b13740640 1 -- 172.21.15.163:0/3323313998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0c0aa850 con 0x7f5b0c072d20 2024-05-08T06:36:53.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f5b13740640 1 -- 172.21.15.163:0/3323313998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b0c0aae10 con 0x7f5b0c072d20 2024-05-08T06:36:53.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f5b09ffb640 1 -- 172.21.15.163:0/3323313998 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b00005ce0 con 0x7f5b0c072d20 2024-05-08T06:36:53.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f5b09ffb640 1 -- 172.21.15.163:0/3323313998 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00017070 con 0x7f5b0c072d20 2024-05-08T06:36:53.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7f3651d4d640 1 Processor -- start 2024-05-08T06:36:53.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.669+0000 7f9e0bfff640 1 -- start start 2024-05-08T06:36:53.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f9e0bfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df40051c0 0x7f9e0c15e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f9e0bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e0c158d30 0x7f9e0c1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f9e0bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e0c1fce70 0x7f9e0c1fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f9e0bfff640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c163a70 con 0x7f9df40051c0 2024-05-08T06:36:53.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f9e0bfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c073b70 con 0x7f9e0c1fce70 2024-05-08T06:36:53.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f9e0bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9e0c163bf0 con 0x7f9e0c158d30 2024-05-08T06:36:53.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f9e0affd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df40051c0 0x7f9e0c15e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7f9e0affd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df40051c0 0x7f9e0c15e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44742/0 (socket says 172.21.15.163:44742) 2024-05-08T06:36:53.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7f9e0affd640 1 -- 172.21.15.163:0/3197111561 learned_addr learned my addr 172.21.15.163:0/3197111561 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:53.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7f9e0affd640 1 -- 172.21.15.163:0/3197111561 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e0c158d30 msgr2=0x7f9e0c1597a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T06:36:53.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7f9e0affd640 1 --2- 172.21.15.163:0/3197111561 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e0c158d30 0x7f9e0c1597a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7f9e0affd640 1 -- 172.21.15.163:0/3197111561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e0c1fce70 msgr2=0x7f9e0c1fd2f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T06:36:53.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7f9e0affd640 1 --2- 172.21.15.163:0/3197111561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e0c1fce70 0x7f9e0c1fd2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7f9e0affd640 1 -- 172.21.15.163:0/3197111561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0c157cb0 con 0x7f9df40051c0 2024-05-08T06:36:53.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f2b7effd640 1 -- 172.21.15.163:0/603239468 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2b8001a070 con 0x7f2b90166d90 2024-05-08T06:36:53.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7fa1a3fff640 1 Processor -- start 2024-05-08T06:36:53.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7fa1a3fff640 1 -- start start 2024-05-08T06:36:53.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a40b5010 0x7fa1a406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1a40cd650 0x7fa1a40cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1a414cfe0 0x7fa1a41fc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7fa1a3fff640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a406e590 con 0x7fa1a40cd650 2024-05-08T06:36:53.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7fa1a3fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a406fcd0 con 0x7fa1a40b5010 2024-05-08T06:36:53.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7fa1a3fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a406e710 con 0x7fa1a414cfe0 2024-05-08T06:36:53.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7effa2ffd640 1 -- 172.21.15.163:0/4181133707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effa40b4cb0 msgr2=0x7effa41e5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7effa2ffd640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effa40b4cb0 0x7effa41e5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7effa2ffd640 1 -- 172.21.15.163:0/4181133707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effa40d1aa0 msgr2=0x7effa41e6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.670+0000 7effa2ffd640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effa40d1aa0 0x7effa41e6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7effa2ffd640 1 -- 172.21.15.163:0/4181133707 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa41f99b0 con 0x7effa4073ed0 2024-05-08T06:36:53.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7effa2ffd640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4073ed0 0x7effa4158660 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0097d0 tx=0x7eff8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7effa37fe640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effa40d1aa0 0x7effa41e6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:53.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7effa27fc640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effa40b4cb0 0x7effa41e5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:53.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7f9e0affd640 1 --2- 172.21.15.163:0/3197111561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df40051c0 0x7f9e0c15e420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009f00 tx=0x7f9e0400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1a414cfe0 0x7fa1a41fc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1a414cfe0 0x7fa1a41fc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:48534/0 (socket says 172.21.15.163:48534) 2024-05-08T06:36:53.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7fa1a37fe640 1 -- 172.21.15.163:0/2457557178 learned_addr learned my addr 172.21.15.163:0/2457557178 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:53.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f5af37fe640 1 -- 172.21.15.163:0/3323313998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b0c1fa4c0 con 0x7f5b0c072d20 2024-05-08T06:36:53.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f9df3fff640 1 -- 172.21.15.163:0/3197111561 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04019020 con 0x7f9df40051c0 2024-05-08T06:36:53.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7eff9bfff640 1 -- 172.21.15.163:0/4181133707 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c042020 con 0x7effa4073ed0 2024-05-08T06:36:53.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7eff9bfff640 1 -- 172.21.15.163:0/4181133707 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff8c0308f0 con 0x7effa4073ed0 2024-05-08T06:36:53.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7effa997f640 1 -- 172.21.15.163:0/4181133707 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa41f9c90 con 0x7effa4073ed0 2024-05-08T06:36:53.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.671+0000 7f3651d4d640 1 -- start start 2024-05-08T06:36:53.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f3651d4d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f364c13ed00 0x7f364c15d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f3651d4d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c15b060 0x7f364c142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f3651d4d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13f0e0 0x7f364c1518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f3651d4d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f364c0b5260 con 0x7f364c13ed00 2024-05-08T06:36:53.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f3651d4d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f364c0b50e0 con 0x7f364c13f0e0 2024-05-08T06:36:53.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f3651d4d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f364c0b53e0 con 0x7f364c15b060 2024-05-08T06:36:53.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f9e0bfff640 1 -- 172.21.15.163:0/3197111561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0c157f90 con 0x7f9df40051c0 2024-05-08T06:36:53.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f9e0bfff640 1 -- 172.21.15.163:0/3197111561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e0c158550 con 0x7f9df40051c0 2024-05-08T06:36:53.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7f2b7effd640 1 --2- 172.21.15.163:0/603239468 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2b700763a0 0x7f2b70078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.672+0000 7effa997f640 1 -- 172.21.15.163:0/4181133707 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa41fa250 con 0x7effa4073ed0 2024-05-08T06:36:53.723 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.673+0000 7f9df3fff640 1 -- 172.21.15.163:0/3197111561 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e0400fe10 con 0x7f9df40051c0 2024-05-08T06:36:53.723 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.673+0000 7f2b8e59c640 1 --2- 172.21.15.163:0/603239468 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2b700763a0 0x7f2b70078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.723 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.673+0000 7f2b7effd640 1 -- 172.21.15.163:0/603239468 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2b80097530 con 0x7f2b90166d90 2024-05-08T06:36:53.723 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.673+0000 7eff9bfff640 1 -- 172.21.15.163:0/4181133707 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c037040 con 0x7effa4073ed0 2024-05-08T06:36:53.724 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.673+0000 7f9df3fff640 1 -- 172.21.15.163:0/3197111561 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04003b90 con 0x7f9df40051c0 2024-05-08T06:36:53.724 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.673+0000 7f2b94d12640 1 -- 172.21.15.163:0/603239468 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b74005290 con 0x7f2b90166d90 2024-05-08T06:36:53.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.673+0000 7f365154c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13f0e0 0x7f364c1518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.673+0000 7f365154c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13f0e0 0x7f364c1518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:35950/0 (socket says 172.21.15.163:35950) 2024-05-08T06:36:53.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.673+0000 7f365154c640 1 -- 172.21.15.163:0/628579262 learned_addr learned my addr 172.21.15.163:0/628579262 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:53.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.673+0000 7f365154c640 1 -- 172.21.15.163:0/628579262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c15b060 msgr2=0x7f364c142140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T06:36:53.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.673+0000 7f365154c640 1 --2- 172.21.15.163:0/628579262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c15b060 0x7f364c142140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.673+0000 7f365154c640 1 -- 172.21.15.163:0/628579262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f364c13ed00 msgr2=0x7f364c15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.674+0000 7fa1a2ffd640 1 --2- 172.21.15.163:0/2457557178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a40b5010 0x7fa1a406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.673+0000 7f365154c640 1 --2- 172.21.15.163:0/628579262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f364c13ed00 0x7f364c15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.673+0000 7f365154c640 1 -- 172.21.15.163:0/628579262 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364c14ff60 con 0x7f364c13f0e0 2024-05-08T06:36:53.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.674+0000 7f365154c640 1 --2- 172.21.15.163:0/628579262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13f0e0 0x7f364c1518f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f364c15ec40 tx=0x7f364400cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.674+0000 7fa1a27fc640 1 --2- 172.21.15.163:0/2457557178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1a40cd650 0x7fa1a40cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.674+0000 7f9e0bfff640 1 -- 172.21.15.163:0/3197111561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dec005290 con 0x7f9df40051c0 2024-05-08T06:36:53.727 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.674+0000 7fa1a2ffd640 1 -- 172.21.15.163:0/2457557178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1a414cfe0 msgr2=0x7fa1a41fc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.727 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.674+0000 7fa1a2ffd640 1 --2- 172.21.15.163:0/2457557178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1a414cfe0 0x7fa1a41fc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.727 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.674+0000 7fa1a2ffd640 1 -- 172.21.15.163:0/2457557178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1a40cd650 msgr2=0x7fa1a40cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:53.727 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.674+0000 7fa1a2ffd640 1 --2- 172.21.15.163:0/2457557178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1a40cd650 0x7fa1a40cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:53.727 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.674+0000 7fa1a2ffd640 1 -- 172.21.15.163:0/2457557178 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a4149220 con 0x7fa1a40b5010 2024-05-08T06:36:53.728 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.675+0000 7f2b8e59c640 1 --2- 172.21.15.163:0/603239468 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2b700763a0 0x7f2b70078860 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2b88044000 tx=0x7f2b88009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.728 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.674+0000 7f1721ffb640 1 -- 172.21.15.163:0/1027129708 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1724042070 con 0x7f172c15f770 2024-05-08T06:36:53.728 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.675+0000 7f1721ffb640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f17100763a0 0x7f1710078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.728 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.675+0000 7f1721ffb640 1 -- 172.21.15.163:0/1027129708 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f17240b9070 con 0x7f172c15f770 2024-05-08T06:36:53.728 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.675+0000 7f9df3fff640 1 -- 172.21.15.163:0/3197111561 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9e0401c020 con 0x7f9df40051c0 2024-05-08T06:36:53.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.675+0000 7fa1a2ffd640 1 --2- 172.21.15.163:0/2457557178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a40b5010 0x7fa1a406eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa194031000 tx=0x7fa1940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.675+0000 7eff99ffb640 1 -- 172.21.15.163:0/4181133707 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff7c005290 con 0x7effa4073ed0 2024-05-08T06:36:53.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.676+0000 7f173174b640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f17100763a0 0x7f1710078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.676+0000 7f9df3fff640 1 --2- 172.21.15.163:0/3197111561 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9df80763d0 0x7f9df8078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.676+0000 7f9df3fff640 1 -- 172.21.15.163:0/3197111561 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9e040986e0 con 0x7f9df40051c0 2024-05-08T06:36:53.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.676+0000 7f9e0a7fc640 1 --2- 172.21.15.163:0/3197111561 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9df80763d0 0x7f9df8078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.676+0000 7eff9bfff640 1 -- 172.21.15.163:0/4181133707 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7eff8c045020 con 0x7effa4073ed0 2024-05-08T06:36:53.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.677+0000 7f9e0a7fc640 1 --2- 172.21.15.163:0/3197111561 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9df80763d0 0x7f9df8078890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c0b3080 tx=0x7f9dfc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.677+0000 7f173174b640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f17100763a0 0x7f1710078860 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f171c0017c0 tx=0x7f171c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.677+0000 7eff9bfff640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7eff84076320 0x7eff840787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.677+0000 7f5b09ffb640 1 -- 172.21.15.163:0/3323313998 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5b0001a070 con 0x7f5b0c072d20 2024-05-08T06:36:53.732 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.677+0000 7effa27fc640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7eff84076320 0x7eff840787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.732 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.677+0000 7f5b09ffb640 1 --2- 172.21.15.163:0/3323313998 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5af80764d0 0x7f5af8078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.732 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.677+0000 7effa27fc640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7eff84076320 0x7eff840787e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7effa40708d0 tx=0x7eff94008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.733 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.677+0000 7f5b09ffb640 1 -- 172.21.15.163:0/3323313998 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5b00096d00 con 0x7f5b0c072d20 2024-05-08T06:36:53.733 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.677+0000 7f5b10cdb640 1 --2- 172.21.15.163:0/3323313998 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5af80764d0 0x7f5af8078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.733 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.678+0000 7f5b10cdb640 1 --2- 172.21.15.163:0/3323313998 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5af80764d0 0x7f5af8078990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5afc034000 tx=0x7f5afc02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.733 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.678+0000 7f2b7effd640 1 -- 172.21.15.163:0/603239468 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2b80060e70 con 0x7f2b90166d90 2024-05-08T06:36:53.733 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.680+0000 7f5b09ffb640 1 -- 172.21.15.163:0/3323313998 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5b000600f0 con 0x7f5b0c072d20 2024-05-08T06:36:53.734 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.680+0000 7eff9bfff640 1 -- 172.21.15.163:0/4181133707 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7eff8c030ce0 con 0x7effa4073ed0 2024-05-08T06:36:53.734 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.683+0000 7fa18b7fe640 1 -- 172.21.15.163:0/2457557178 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194042070 con 0x7fa1a40b5010 2024-05-08T06:36:53.734 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.683+0000 7fa18b7fe640 1 -- 172.21.15.163:0/2457557178 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa194030750 con 0x7fa1a40b5010 2024-05-08T06:36:53.734 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.683+0000 7f1721ffb640 1 -- 172.21.15.163:0/1027129708 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f17240823f0 con 0x7f172c15f770 2024-05-08T06:36:53.735 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.683+0000 7f36497fa640 1 -- 172.21.15.163:0/628579262 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644017070 con 0x7f364c13f0e0 2024-05-08T06:36:53.735 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.683+0000 7f3651d4d640 1 -- 172.21.15.163:0/628579262 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364c13f8f0 con 0x7f364c13f0e0 2024-05-08T06:36:53.735 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.683+0000 7f3651d4d640 1 -- 172.21.15.163:0/628579262 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f364c13feb0 con 0x7f364c13f0e0 2024-05-08T06:36:53.735 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.683+0000 7f36497fa640 1 -- 172.21.15.163:0/628579262 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3644005d40 con 0x7f364c13f0e0 2024-05-08T06:36:53.735 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.684+0000 7fa18b7fe640 1 -- 172.21.15.163:0/2457557178 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194037070 con 0x7fa1a40b5010 2024-05-08T06:36:53.736 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.684+0000 7f36497fa640 1 -- 172.21.15.163:0/628579262 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364400a850 con 0x7f364c13f0e0 2024-05-08T06:36:53.736 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.684+0000 7fa1a3fff640 1 -- 172.21.15.163:0/2457557178 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a41fd090 con 0x7fa1a40b5010 2024-05-08T06:36:53.736 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.684+0000 7fa1a3fff640 1 -- 172.21.15.163:0/2457557178 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41e88f0 con 0x7fa1a40b5010 2024-05-08T06:36:53.736 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.685+0000 7f3651d4d640 1 -- 172.21.15.163:0/628579262 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f362c005290 con 0x7f364c13f0e0 2024-05-08T06:36:53.737 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.686+0000 7f9df3fff640 1 -- 172.21.15.163:0/3197111561 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9e04023020 con 0x7f9df40051c0 2024-05-08T06:36:53.737 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.688+0000 7fa18b7fe640 1 -- 172.21.15.163:0/2457557178 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa194045070 con 0x7fa1a40b5010 2024-05-08T06:36:53.737 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.688+0000 7fa1a3fff640 1 -- 172.21.15.163:0/2457557178 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a406a3d0 con 0x7fa1a40b5010 2024-05-08T06:36:53.737 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.688+0000 7fa18b7fe640 1 --2- 172.21.15.163:0/2457557178 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa1900762d0 0x7fa190078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.737 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.689+0000 7f36497fa640 1 -- 172.21.15.163:0/628579262 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f364401a070 con 0x7f364c13f0e0 2024-05-08T06:36:53.738 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.689+0000 7fa1a27fc640 1 --2- 172.21.15.163:0/2457557178 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa1900762d0 0x7fa190078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.738 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.690+0000 7f36497fa640 1 --2- 172.21.15.163:0/628579262 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f36300763d0 0x7f3630078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:53.738 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.690+0000 7f36497fa640 1 -- 172.21.15.163:0/628579262 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3644096ce0 con 0x7f364c13f0e0 2024-05-08T06:36:53.738 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.690+0000 7f3650d4b640 1 --2- 172.21.15.163:0/628579262 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f36300763d0 0x7f3630078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:53.739 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.690+0000 7fa18b7fe640 1 -- 172.21.15.163:0/2457557178 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa1940b79b0 con 0x7fa1a40b5010 2024-05-08T06:36:53.739 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.691+0000 7f3650d4b640 1 --2- 172.21.15.163:0/628579262 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f36300763d0 0x7f3630078890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3640032000 tx=0x7f3640009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.739 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.691+0000 7f36497fa640 1 -- 172.21.15.163:0/628579262 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f364403a080 con 0x7f364c13f0e0 2024-05-08T06:36:53.739 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.691+0000 7fa1a27fc640 1 --2- 172.21.15.163:0/2457557178 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa1900762d0 0x7fa190078790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa19c007fd0 tx=0x7fa19c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:53.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.693+0000 7eff9bfff640 1 -- 172.21.15.163:0/4181133707 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7eff8c04c030 con 0x7effa4073ed0 2024-05-08T06:36:53.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:53.696+0000 7fa18b7fe640 1 -- 172.21.15.163:0/2457557178 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa194080c20 con 0x7fa1a40b5010 2024-05-08T06:36:54.046 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.047+0000 7fb841e59640 1 -- 172.21.15.163:0/495153219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb828005580 con 0x7fb83c16c2b0 2024-05-08T06:36:54.046 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.047+0000 7fb823fff640 1 -- 172.21.15.163:0/495153219 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb8340643e0 con 0x7fb83c16c2b0 2024-05-08T06:36:54.047 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2024-05-08T06:36:54.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.051+0000 7fb821ffb640 1 -- 172.21.15.163:0/495153219 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb82407a950 msgr2=0x7fb82407ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.051+0000 7fb821ffb640 1 --2- 172.21.15.163:0/495153219 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb82407a950 0x7fb82407ce10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb82c037000 tx=0x7fb82c00b040 comp rx=0 tx=0).stop 2024-05-08T06:36:54.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.051+0000 7fb821ffb640 1 -- 172.21.15.163:0/495153219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb83c16c2b0 msgr2=0x7fb83c08c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.051+0000 7fb821ffb640 1 --2- 172.21.15.163:0/495153219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb83c16c2b0 0x7fb83c08c2a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb83400caf0 tx=0x7fb83400a060 comp rx=0 tx=0).stop 2024-05-08T06:36:54.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.051+0000 7fb821ffb640 1 -- 172.21.15.163:0/495153219 shutdown_connections 2024-05-08T06:36:54.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.051+0000 7fb821ffb640 1 --2- 172.21.15.163:0/495153219 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb82407a950 0x7fb82407ce10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.051+0000 7fb821ffb640 1 --2- 172.21.15.163:0/495153219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb83c0af5f0 0x7fb83c0afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.051+0000 7fb821ffb640 1 --2- 172.21.15.163:0/495153219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb83c16c2b0 0x7fb83c08c2a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.051+0000 7fb821ffb640 1 --2- 172.21.15.163:0/495153219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb83c0b6080 0x7fb83c165790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.051+0000 7fb821ffb640 1 -- 172.21.15.163:0/495153219 >> 172.21.15.163:0/495153219 conn(0x7fb83c06c320 msgr2=0x7fb83c0adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:54.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.052+0000 7fb821ffb640 1 -- 172.21.15.163:0/495153219 shutdown_connections 2024-05-08T06:36:54.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.052+0000 7fb821ffb640 1 -- 172.21.15.163:0/495153219 wait complete. 2024-05-08T06:36:54.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-05-08T06:36:54.069 DEBUG:teuthology.parallel:result is None 2024-05-08T06:36:54.090 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.091+0000 7f3651d4d640 1 -- 172.21.15.163:0/628579262 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f362c005850 con 0x7f364c13f0e0 2024-05-08T06:36:54.091 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.092+0000 7f36497fa640 1 -- 172.21.15.163:0/628579262 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f36440606b0 con 0x7f364c13f0e0 2024-05-08T06:36:54.091 INFO:teuthology.orchestra.run.smithi163.stdout:55834574856 2024-05-08T06:36:54.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.094+0000 7f3651d4d640 1 -- 172.21.15.163:0/628579262 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f36300763d0 msgr2=0x7f3630078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.094+0000 7f3651d4d640 1 --2- 172.21.15.163:0/628579262 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f36300763d0 0x7f3630078890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3640032000 tx=0x7f3640009030 comp rx=0 tx=0).stop 2024-05-08T06:36:54.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.095+0000 7f3651d4d640 1 -- 172.21.15.163:0/628579262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13f0e0 msgr2=0x7f364c1518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.095+0000 7f3651d4d640 1 --2- 172.21.15.163:0/628579262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13f0e0 0x7f364c1518f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f364c15ec40 tx=0x7f364400cb90 comp rx=0 tx=0).stop 2024-05-08T06:36:54.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.095+0000 7f3651d4d640 1 -- 172.21.15.163:0/628579262 shutdown_connections 2024-05-08T06:36:54.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.095+0000 7f3651d4d640 1 --2- 172.21.15.163:0/628579262 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f36300763d0 0x7f3630078890 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.095+0000 7f3651d4d640 1 --2- 172.21.15.163:0/628579262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13f0e0 0x7f364c1518f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.095+0000 7f3651d4d640 1 --2- 172.21.15.163:0/628579262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c15b060 0x7f364c142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.095+0000 7f3651d4d640 1 --2- 172.21.15.163:0/628579262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f364c13ed00 0x7f364c15d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.095+0000 7f3651d4d640 1 -- 172.21.15.163:0/628579262 >> 172.21.15.163:0/628579262 conn(0x7f364c0d0c20 msgr2=0x7f364c0b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:54.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.095+0000 7f3651d4d640 1 -- 172.21.15.163:0/628579262 shutdown_connections 2024-05-08T06:36:54.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.095+0000 7f3651d4d640 1 -- 172.21.15.163:0/628579262 wait complete. 2024-05-08T06:36:54.104 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.105+0000 7f5af37fe640 1 -- 172.21.15.163:0/3323313998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5b0c167e90 con 0x7f5b0c072d20 2024-05-08T06:36:54.105 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.106+0000 7f5b09ffb640 1 -- 172.21.15.163:0/3323313998 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5b000643f0 con 0x7f5b0c072d20 2024-05-08T06:36:54.105 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2024-05-08T06:36:54.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.108+0000 7f2b94d12640 1 -- 172.21.15.163:0/603239468 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2b74005580 con 0x7f2b90166d90 2024-05-08T06:36:54.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.108+0000 7f5af37fe640 1 -- 172.21.15.163:0/3323313998 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5af80764d0 msgr2=0x7f5af8078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.108+0000 7f5af37fe640 1 --2- 172.21.15.163:0/3323313998 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5af80764d0 0x7f5af8078990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5afc034000 tx=0x7f5afc02f000 comp rx=0 tx=0).stop 2024-05-08T06:36:54.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.108+0000 7f5af37fe640 1 -- 172.21.15.163:0/3323313998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b0c072d20 msgr2=0x7f5b0c161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.108+0000 7f5af37fe640 1 --2- 172.21.15.163:0/3323313998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b0c072d20 0x7f5b0c161240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5b00007fd0 tx=0x7f5b0000c9e0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.109+0000 7f5af37fe640 1 -- 172.21.15.163:0/3323313998 shutdown_connections 2024-05-08T06:36:54.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.109+0000 7f5af37fe640 1 --2- 172.21.15.163:0/3323313998 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5af80764d0 0x7f5af8078990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.109+0000 7f5af37fe640 1 --2- 172.21.15.163:0/3323313998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b0c149690 0x7f5b0c0cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.109+0000 7f5af37fe640 1 --2- 172.21.15.163:0/3323313998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b0c146790 0x7f5b0c161780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.109+0000 7f5af37fe640 1 --2- 172.21.15.163:0/3323313998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b0c072d20 0x7f5b0c161240 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.109+0000 7f2b7effd640 1 -- 172.21.15.163:0/603239468 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2b80060770 con 0x7f2b90166d90 2024-05-08T06:36:54.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.109+0000 7f5af37fe640 1 -- 172.21.15.163:0/3323313998 >> 172.21.15.163:0/3323313998 conn(0x7f5b0c0682f0 msgr2=0x7f5b0c167020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:54.109 INFO:teuthology.orchestra.run.smithi163.stdout:55834574857 2024-05-08T06:36:54.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.109+0000 7f5af37fe640 1 -- 172.21.15.163:0/3323313998 shutdown_connections 2024-05-08T06:36:54.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.109+0000 7f5af37fe640 1 -- 172.21.15.163:0/3323313998 wait complete. 2024-05-08T06:36:54.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2024-05-08T06:36:54.111 DEBUG:teuthology.parallel:result is None 2024-05-08T06:36:54.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.111+0000 7f2b94d12640 1 -- 172.21.15.163:0/603239468 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2b700763a0 msgr2=0x7f2b70078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.112+0000 7f2b94d12640 1 --2- 172.21.15.163:0/603239468 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2b700763a0 0x7f2b70078860 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2b88044000 tx=0x7f2b88009000 comp rx=0 tx=0).stop 2024-05-08T06:36:54.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.112+0000 7f2b94d12640 1 -- 172.21.15.163:0/603239468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b90166d90 msgr2=0x7f2b900b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.112+0000 7f2b94d12640 1 --2- 172.21.15.163:0/603239468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b90166d90 0x7f2b900b1710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000e830 tx=0x7f2b8000ecf0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.112+0000 7f2b94d12640 1 -- 172.21.15.163:0/603239468 shutdown_connections 2024-05-08T06:36:54.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.112+0000 7f2b94d12640 1 --2- 172.21.15.163:0/603239468 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2b700763a0 0x7f2b70078860 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.112+0000 7f2b94d12640 1 --2- 172.21.15.163:0/603239468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b90166d90 0x7f2b900b1710 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.112+0000 7f2b94d12640 1 --2- 172.21.15.163:0/603239468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b900d17a0 0x7f2b9015a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.112+0000 7f2b94d12640 1 --2- 172.21.15.163:0/603239468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b900b4460 0x7f2b9015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.112+0000 7f2b94d12640 1 -- 172.21.15.163:0/603239468 >> 172.21.15.163:0/603239468 conn(0x7f2b9006c740 msgr2=0x7f2b900b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:54.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.112+0000 7f2b94d12640 1 -- 172.21.15.163:0/603239468 shutdown_connections 2024-05-08T06:36:54.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.112+0000 7f2b94d12640 1 -- 172.21.15.163:0/603239468 wait complete. 2024-05-08T06:36:54.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.118+0000 7f9e0bfff640 1 -- 172.21.15.163:0/3197111561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9dec005850 con 0x7f9df40051c0 2024-05-08T06:36:54.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.118+0000 7eff99ffb640 1 -- 172.21.15.163:0/4181133707 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7eff7c005580 con 0x7effa4073ed0 2024-05-08T06:36:54.118 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.119+0000 7f9df3fff640 1 -- 172.21.15.163:0/3197111561 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9e04010040 con 0x7f9df40051c0 2024-05-08T06:36:54.118 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2024-05-08T06:36:54.118 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.119+0000 7eff9bfff640 1 -- 172.21.15.163:0/4181133707 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7eff8c081b90 con 0x7effa4073ed0 2024-05-08T06:36:54.119 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2024-05-08T06:36:54.120 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.121+0000 7f9e0bfff640 1 -- 172.21.15.163:0/3197111561 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9df80763d0 msgr2=0x7f9df8078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.121+0000 7f9e0bfff640 1 --2- 172.21.15.163:0/3197111561 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9df80763d0 0x7f9df8078890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c0b3080 tx=0x7f9dfc009040 comp rx=0 tx=0).stop 2024-05-08T06:36:54.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.121+0000 7f9e0bfff640 1 -- 172.21.15.163:0/3197111561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df40051c0 msgr2=0x7f9e0c15e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.121+0000 7f9e0bfff640 1 --2- 172.21.15.163:0/3197111561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df40051c0 0x7f9e0c15e420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009f00 tx=0x7f9e0400f580 comp rx=0 tx=0).stop 2024-05-08T06:36:54.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.121+0000 7f9e0bfff640 1 -- 172.21.15.163:0/3197111561 shutdown_connections 2024-05-08T06:36:54.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.121+0000 7f9e0bfff640 1 --2- 172.21.15.163:0/3197111561 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9df80763d0 0x7f9df8078890 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.121+0000 7f9e0bfff640 1 --2- 172.21.15.163:0/3197111561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e0c1fce70 0x7f9e0c1fd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.121+0000 7f9e0bfff640 1 --2- 172.21.15.163:0/3197111561 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e0c158d30 0x7f9e0c1597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.121+0000 7f9e0bfff640 1 --2- 172.21.15.163:0/3197111561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9df40051c0 0x7f9e0c15e420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.121+0000 7f9e0bfff640 1 -- 172.21.15.163:0/3197111561 >> 172.21.15.163:0/3197111561 conn(0x7f9e0c0b32b0 msgr2=0x7f9e0c164b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:54.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7f17339af640 1 -- 172.21.15.163:0/1027129708 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f172c0d2f40 con 0x7f172c15f770 2024-05-08T06:36:54.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7f9e0bfff640 1 -- 172.21.15.163:0/3197111561 shutdown_connections 2024-05-08T06:36:54.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7f9e0bfff640 1 -- 172.21.15.163:0/3197111561 wait complete. 2024-05-08T06:36:54.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7eff99ffb640 1 -- 172.21.15.163:0/4181133707 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7eff84076320 msgr2=0x7eff840787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7eff99ffb640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7eff84076320 0x7eff840787e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7effa40708d0 tx=0x7eff94008040 comp rx=0 tx=0).stop 2024-05-08T06:36:54.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7eff99ffb640 1 -- 172.21.15.163:0/4181133707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4073ed0 msgr2=0x7effa4158660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7eff99ffb640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4073ed0 0x7effa4158660 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0097d0 tx=0x7eff8c0046c0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7eff99ffb640 1 -- 172.21.15.163:0/4181133707 shutdown_connections 2024-05-08T06:36:54.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7eff99ffb640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7eff84076320 0x7eff840787e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7eff99ffb640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effa40d1aa0 0x7effa41e6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7eff99ffb640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effa40b4cb0 0x7effa41e5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7eff99ffb640 1 --2- 172.21.15.163:0/4181133707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4073ed0 0x7effa4158660 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7eff99ffb640 1 -- 172.21.15.163:0/4181133707 >> 172.21.15.163:0/4181133707 conn(0x7effa406c740 msgr2=0x7effa415c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:54.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7f1721ffb640 1 -- 172.21.15.163:0/1027129708 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f17240866f0 con 0x7f172c15f770 2024-05-08T06:36:54.124 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2024-05-08T06:36:54.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7eff99ffb640 1 -- 172.21.15.163:0/4181133707 shutdown_connections 2024-05-08T06:36:54.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.122+0000 7eff99ffb640 1 -- 172.21.15.163:0/4181133707 wait complete. 2024-05-08T06:36:54.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.126+0000 7f17339af640 1 -- 172.21.15.163:0/1027129708 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f17100763a0 msgr2=0x7f1710078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.126+0000 7f17339af640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f17100763a0 0x7f1710078860 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f171c0017c0 tx=0x7f171c009040 comp rx=0 tx=0).stop 2024-05-08T06:36:54.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.126+0000 7f17339af640 1 -- 172.21.15.163:0/1027129708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c15f770 msgr2=0x7f172c0d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.126+0000 7f17339af640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c15f770 0x7f172c0d1cb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1724014a00 tx=0x7f1724004a40 comp rx=0 tx=0).stop 2024-05-08T06:36:54.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.126+0000 7f17339af640 1 -- 172.21.15.163:0/1027129708 shutdown_connections 2024-05-08T06:36:54.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.126+0000 7f17339af640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f17100763a0 0x7f1710078860 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.126+0000 7f17339af640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c15f770 0x7f172c0d1cb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.126+0000 7f17339af640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f172c15f390 0x7f172c159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.126+0000 7f17339af640 1 --2- 172.21.15.163:0/1027129708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f172c0d0780 0x7f172c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.126+0000 7f17339af640 1 -- 172.21.15.163:0/1027129708 >> 172.21.15.163:0/1027129708 conn(0x7f172c06d300 msgr2=0x7f172c15fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:54.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.126+0000 7f17339af640 1 -- 172.21.15.163:0/1027129708 shutdown_connections 2024-05-08T06:36:54.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.126+0000 7f17339af640 1 -- 172.21.15.163:0/1027129708 wait complete. 2024-05-08T06:36:54.127 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-05-08T06:36:54.128 DEBUG:teuthology.parallel:result is None 2024-05-08T06:36:54.129 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2024-05-08T06:36:54.129 DEBUG:teuthology.parallel:result is None 2024-05-08T06:36:54.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.130+0000 7fa1a3fff640 1 -- 172.21.15.163:0/2457557178 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa1a40b4aa0 con 0x7fa1a40b5010 2024-05-08T06:36:54.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.131+0000 7fa18b7fe640 1 -- 172.21.15.163:0/2457557178 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa194084f20 con 0x7fa1a40b5010 2024-05-08T06:36:54.130 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2024-05-08T06:36:54.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.133+0000 7fa1a3fff640 1 -- 172.21.15.163:0/2457557178 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa1900762d0 msgr2=0x7fa190078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.133+0000 7fa1a3fff640 1 --2- 172.21.15.163:0/2457557178 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa1900762d0 0x7fa190078790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa19c007fd0 tx=0x7fa19c009040 comp rx=0 tx=0).stop 2024-05-08T06:36:54.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.133+0000 7fa1a3fff640 1 -- 172.21.15.163:0/2457557178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a40b5010 msgr2=0x7fa1a406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.133+0000 7fa1a3fff640 1 --2- 172.21.15.163:0/2457557178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a40b5010 0x7fa1a406eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa194031000 tx=0x7fa1940046c0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.133+0000 7fa1a3fff640 1 -- 172.21.15.163:0/2457557178 shutdown_connections 2024-05-08T06:36:54.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.133+0000 7fa1a3fff640 1 --2- 172.21.15.163:0/2457557178 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa1900762d0 0x7fa190078790 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.133+0000 7fa1a3fff640 1 --2- 172.21.15.163:0/2457557178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1a414cfe0 0x7fa1a41fc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.133+0000 7fa1a3fff640 1 --2- 172.21.15.163:0/2457557178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1a40cd650 0x7fa1a40cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.133+0000 7fa1a3fff640 1 --2- 172.21.15.163:0/2457557178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1a40b5010 0x7fa1a406eb30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.133+0000 7fa1a3fff640 1 -- 172.21.15.163:0/2457557178 >> 172.21.15.163:0/2457557178 conn(0x7fa1a40d0c20 msgr2=0x7fa1a4149750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:54.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.133+0000 7fa1a3fff640 1 -- 172.21.15.163:0/2457557178 shutdown_connections 2024-05-08T06:36:54.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.133+0000 7fa1a3fff640 1 -- 172.21.15.163:0/2457557178 wait complete. 2024-05-08T06:36:54.138 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-05-08T06:36:54.138 DEBUG:teuthology.parallel:result is None 2024-05-08T06:36:54.140 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-05-08T06:36:54.140 DEBUG:teuthology.parallel:result is None 2024-05-08T06:36:54.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-05-08T06:36:54.143 DEBUG:teuthology.parallel:result is None 2024-05-08T06:36:54.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-05-08T06:36:54.145 DEBUG:teuthology.parallel:result is None 2024-05-08T06:36:54.145 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T06:36:54.145 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:36:54.285 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.286+0000 7f685c3b6640 1 Processor -- start 2024-05-08T06:36:54.285 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.286+0000 7f685c3b6640 1 -- start start 2024-05-08T06:36:54.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f685c3b6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f685415b8a0 0x7f68540725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f685c3b6640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68540b5140 con 0x7f685406f090 2024-05-08T06:36:54.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f685c3b6640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68540b48b0 con 0x7f68540b54d0 2024-05-08T06:36:54.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f685c3b6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f68540af910 con 0x7f685415b8a0 2024-05-08T06:36:54.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f685a953640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f685406f090 0x7f68540afda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57184/0 (socket says 172.21.15.163:57184) 2024-05-08T06:36:54.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f685a953640 1 -- 172.21.15.163:0/4172303168 learned_addr learned my addr 172.21.15.163:0/4172303168 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:54.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f685a152640 1 --2- 172.21.15.163:0/4172303168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f685415b8a0 0x7f68540725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1339836747 0 0) 0x7f68540b5140 con 0x7f685406f090 2024-05-08T06:36:54.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6840003520 con 0x7f685406f090 2024-05-08T06:36:54.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2796646124 0 0) 0x7f68540b48b0 con 0x7f68540b54d0 2024-05-08T06:36:54.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68540b5140 con 0x7f68540b54d0 2024-05-08T06:36:54.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2298825794 0 0) 0x7f6840003520 con 0x7f685406f090 2024-05-08T06:36:54.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f685415b8a0 msgr2=0x7f68540725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:54.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f6859150640 1 --2- 172.21.15.163:0/4172303168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f685415b8a0 0x7f68540725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 >> v1:172.21.15.192:6789/0 conn(0x7f68540b54d0 legacy=0x7f68540b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68540b4b80 con 0x7f685406f090 2024-05-08T06:36:54.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f684c001a70 con 0x7f685406f090 2024-05-08T06:36:54.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 >> v1:172.21.15.163:6789/0 conn(0x7f685406f090 legacy=0x7f68540afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f6859150640 1 --2- 172.21.15.163:0/4172303168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6840003c60 0x7f6840004060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f6859150640 1 --2- 172.21.15.163:0/4172303168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6840004800 0x7f6840004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.287+0000 7f6859150640 1 --2- 172.21.15.163:0/4172303168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f685415b8a0 0x7f6840005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.288+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f68540af910 con 0x7f6840004800 2024-05-08T06:36:54.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.288+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f684000dd90 con 0x7f6840003c60 2024-05-08T06:36:54.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.288+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f684000df90 con 0x7f685415b8a0 2024-05-08T06:36:54.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.288+0000 7f6859951640 1 --2- 172.21.15.163:0/4172303168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6840004800 0x7f6840004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.288+0000 7f685a152640 1 --2- 172.21.15.163:0/4172303168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6840003c60 0x7f6840004060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.288+0000 7f685a953640 1 --2- 172.21.15.163:0/4172303168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f685415b8a0 0x7f6840005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.288+0000 7f6859951640 1 -- 172.21.15.163:0/4172303168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f685415b8a0 msgr2=0x7f6840005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:54.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.288+0000 7f6859951640 1 --2- 172.21.15.163:0/4172303168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f685415b8a0 0x7f6840005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.288+0000 7f6859951640 1 -- 172.21.15.163:0/4172303168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6840003c60 msgr2=0x7f6840004060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:54.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.288+0000 7f6859951640 1 --2- 172.21.15.163:0/4172303168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6840003c60 0x7f6840004060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.288+0000 7f6859951640 1 -- 172.21.15.163:0/4172303168 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6840003520 con 0x7f6840004800 2024-05-08T06:36:54.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.289+0000 7f6859951640 1 --2- 172.21.15.163:0/4172303168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6840004800 0x7f6840004c80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6844001900 tx=0x7f6840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a89b36a829eaf57 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:54.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.289+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844001520 con 0x7f6840004800 2024-05-08T06:36:54.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.289+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f684402acd0 con 0x7f6840004800 2024-05-08T06:36:54.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.289+0000 7f6859150640 1 -- 172.21.15.163:0/4172303168 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844005310 con 0x7f6840004800 2024-05-08T06:36:54.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.289+0000 7f685c3b6640 1 -- 172.21.15.163:0/4172303168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6840004800 msgr2=0x7f6840004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.289+0000 7f685c3b6640 1 --2- 172.21.15.163:0/4172303168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6840004800 0x7f6840004c80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6844001900 tx=0x7f6840005830 comp rx=0 tx=0).stop 2024-05-08T06:36:54.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.289+0000 7f685a953640 1 -- 172.21.15.163:0/4172303168 reap_dead start 2024-05-08T06:36:54.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.289+0000 7f685c3b6640 1 -- 172.21.15.163:0/4172303168 shutdown_connections 2024-05-08T06:36:54.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.289+0000 7f685c3b6640 1 -- 172.21.15.163:0/4172303168 >> 172.21.15.163:0/4172303168 conn(0x7f685406c500 msgr2=0x7f685406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:54.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.289+0000 7f685c3b6640 1 -- 172.21.15.163:0/4172303168 shutdown_connections 2024-05-08T06:36:54.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.289+0000 7f685c3b6640 1 -- 172.21.15.163:0/4172303168 wait complete. 2024-05-08T06:36:54.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.290+0000 7f685c3b6640 1 Processor -- start 2024-05-08T06:36:54.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.291+0000 7f685c3b6640 1 -- start start 2024-05-08T06:36:54.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.291+0000 7f685c3b6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6840004800 0x7f685415e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.291+0000 7f685c3b6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68540cdb20 0x7f685414e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.291+0000 7f685c3b6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68541542b0 0x7f6854154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.291+0000 7f685c3b6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f68540b04b0 con 0x7f68540cdb20 2024-05-08T06:36:54.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.291+0000 7f685c3b6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f68540af5a0 con 0x7f6840004800 2024-05-08T06:36:54.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.291+0000 7f685c3b6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f68540b0630 con 0x7f68541542b0 2024-05-08T06:36:54.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.291+0000 7f6859951640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68540cdb20 0x7f685414e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.291+0000 7f6859951640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68540cdb20 0x7f685414e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44764/0 (socket says 172.21.15.163:44764) 2024-05-08T06:36:54.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.291+0000 7f6859951640 1 -- 172.21.15.163:0/3188441956 learned_addr learned my addr 172.21.15.163:0/3188441956 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:54.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.291+0000 7f685a953640 1 --2- 172.21.15.163:0/3188441956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68541542b0 0x7f6854154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.291+0000 7f685a152640 1 --2- 172.21.15.163:0/3188441956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6840004800 0x7f685415e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.292+0000 7f6859951640 1 -- 172.21.15.163:0/3188441956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68541542b0 msgr2=0x7f6854154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.292+0000 7f6859951640 1 --2- 172.21.15.163:0/3188441956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68541542b0 0x7f6854154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.292+0000 7f6859951640 1 -- 172.21.15.163:0/3188441956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6840004800 msgr2=0x7f685415e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.292+0000 7f6859951640 1 --2- 172.21.15.163:0/3188441956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6840004800 0x7f685415e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.292+0000 7f6859951640 1 -- 172.21.15.163:0/3188441956 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68540ce020 con 0x7f68540cdb20 2024-05-08T06:36:54.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.292+0000 7f6859951640 1 --2- 172.21.15.163:0/3188441956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68540cdb20 0x7f685414e420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6844005040 tx=0x7f68440012f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:54.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.292+0000 7f684affd640 1 -- 172.21.15.163:0/3188441956 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844001520 con 0x7f68540cdb20 2024-05-08T06:36:54.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.292+0000 7f684affd640 1 -- 172.21.15.163:0/3188441956 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68440075e0 con 0x7f68540cdb20 2024-05-08T06:36:54.298 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.292+0000 7f684affd640 1 -- 172.21.15.163:0/3188441956 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68440069d0 con 0x7f68540cdb20 2024-05-08T06:36:54.298 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.292+0000 7f685c3b6640 1 -- 172.21.15.163:0/3188441956 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68541529b0 con 0x7f68540cdb20 2024-05-08T06:36:54.298 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.292+0000 7f685c3b6640 1 -- 172.21.15.163:0/3188441956 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685415eaf0 con 0x7f68540cdb20 2024-05-08T06:36:54.298 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.293+0000 7f685c3b6640 1 -- 172.21.15.163:0/3188441956 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f685406a550 con 0x7f68540cdb20 2024-05-08T06:36:54.298 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.294+0000 7f684affd640 1 -- 172.21.15.163:0/3188441956 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6844044060 con 0x7f68540cdb20 2024-05-08T06:36:54.298 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.294+0000 7f684affd640 1 --2- 172.21.15.163:0/3188441956 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6838076300 0x7f68380787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.298 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.294+0000 7f685a152640 1 --2- 172.21.15.163:0/3188441956 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6838076300 0x7f68380787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.298 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.294+0000 7f684affd640 1 -- 172.21.15.163:0/3188441956 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f68440b6660 con 0x7f68540cdb20 2024-05-08T06:36:54.298 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.294+0000 7f685a152640 1 --2- 172.21.15.163:0/3188441956 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6838076300 0x7f68380787c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f68540b32c0 tx=0x7f6850008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:54.299 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.298+0000 7f684affd640 1 -- 172.21.15.163:0/3188441956 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6844080130 con 0x7f68540cdb20 2024-05-08T06:36:54.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.479+0000 7f685c3b6640 1 -- 172.21.15.163:0/3188441956 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6854154c80 con 0x7f6838076300 2024-05-08T06:36:54.484 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.485+0000 7f684affd640 1 -- 172.21.15.163:0/3188441956 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f68500149d0 con 0x7f6838076300 2024-05-08T06:36:54.487 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2024-05-08T06:36:54.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.488+0000 7f685c3b6640 1 -- 172.21.15.163:0/3188441956 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6838076300 msgr2=0x7f68380787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.488+0000 7f685c3b6640 1 --2- 172.21.15.163:0/3188441956 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6838076300 0x7f68380787c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f68540b32c0 tx=0x7f6850008040 comp rx=0 tx=0).stop 2024-05-08T06:36:54.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.488+0000 7f685c3b6640 1 -- 172.21.15.163:0/3188441956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68540cdb20 msgr2=0x7f685414e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.488+0000 7f685c3b6640 1 --2- 172.21.15.163:0/3188441956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68540cdb20 0x7f685414e420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6844005040 tx=0x7f68440012f0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.488+0000 7f685c3b6640 1 -- 172.21.15.163:0/3188441956 shutdown_connections 2024-05-08T06:36:54.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.488+0000 7f685c3b6640 1 --2- 172.21.15.163:0/3188441956 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6838076300 0x7f68380787c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.488+0000 7f685c3b6640 1 --2- 172.21.15.163:0/3188441956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68541542b0 0x7f6854154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.488+0000 7f685c3b6640 1 --2- 172.21.15.163:0/3188441956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68540cdb20 0x7f685414e420 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.488+0000 7f685c3b6640 1 --2- 172.21.15.163:0/3188441956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6840004800 0x7f685415e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.488+0000 7f685c3b6640 1 -- 172.21.15.163:0/3188441956 >> 172.21.15.163:0/3188441956 conn(0x7f685406c500 msgr2=0x7f68540b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:54.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.488+0000 7f685c3b6640 1 -- 172.21.15.163:0/3188441956 shutdown_connections 2024-05-08T06:36:54.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.488+0000 7f685c3b6640 1 -- 172.21.15.163:0/3188441956 wait complete. 2024-05-08T06:36:54.499 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-05-08T06:36:54.100349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.452350"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109935+0000","last_change":"2024-05-08T06:36:41.664385+0000","last_active":"2024-05-08T06:36:43.109935+0000","last_peered":"2024-05-08T06:36:43.109935+0000","last_clean":"2024-05-08T06:36:43.109935+0000","last_became_active":"2024-05-08T06:36:41.664106+0000","last_became_peered":"2024-05-08T06:36:41.664106+0000","last_unstale":"2024-05-08T06:36:43.109935+0000","last_undegraded":"2024-05-08T06:36:43.109935+0000","last_fullsized":"2024-05-08T06:36:43.109935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:08.774451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649403+0000","last_change":"2024-05-08T06:36:41.647763+0000","last_active":"2024-05-08T06:36:42.649403+0000","last_peered":"2024-05-08T06:36:42.649403+0000","last_clean":"2024-05-08T06:36:42.649403+0000","last_became_active":"2024-05-08T06:36:41.647487+0000","last_became_peered":"2024-05-08T06:36:41.647487+0000","last_unstale":"2024-05-08T06:36:42.649403+0000","last_undegraded":"2024-05-08T06:36:42.649403+0000","last_fullsized":"2024-05-08T06:36:42.649403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108764+0000","last_change":"2024-05-08T06:36:41.649945+0000","last_active":"2024-05-08T06:36:43.108764+0000","last_peered":"2024-05-08T06:36:43.108764+0000","last_clean":"2024-05-08T06:36:43.108764+0000","last_became_active":"2024-05-08T06:36:41.649665+0000","last_became_peered":"2024-05-08T06:36:41.649665+0000","last_unstale":"2024-05-08T06:36:43.108764+0000","last_undegraded":"2024-05-08T06:36:43.108764+0000","last_fullsized":"2024-05-08T06:36:43.108764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109325+0000","last_change":"2024-05-08T06:36:41.656179+0000","last_active":"2024-05-08T06:36:43.109325+0000","last_peered":"2024-05-08T06:36:43.109325+0000","last_clean":"2024-05-08T06:36:43.109325+0000","last_became_active":"2024-05-08T06:36:41.655649+0000","last_became_peered":"2024-05-08T06:36:41.655649+0000","last_unstale":"2024-05-08T06:36:43.109325+0000","last_undegraded":"2024-05-08T06:36:43.109325+0000","last_fullsized":"2024-05-08T06:36:43.109325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:45:07.961465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109228+0000","last_change":"2024-05-08T06:36:41.659928+0000","last_active":"2024-05-08T06:36:43.109228+0000","last_peered":"2024-05-08T06:36:43.109228+0000","last_clean":"2024-05-08T06:36:43.109228+0000","last_became_active":"2024-05-08T06:36:41.659408+0000","last_became_peered":"2024-05-08T06:36:41.659408+0000","last_unstale":"2024-05-08T06:36:43.109228+0000","last_undegraded":"2024-05-08T06:36:43.109228+0000","last_fullsized":"2024-05-08T06:36:43.109228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:14.886723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109064+0000","last_change":"2024-05-08T06:36:41.653658+0000","last_active":"2024-05-08T06:36:43.109064+0000","last_peered":"2024-05-08T06:36:43.109064+0000","last_clean":"2024-05-08T06:36:43.109064+0000","last_became_active":"2024-05-08T06:36:41.653285+0000","last_became_peered":"2024-05-08T06:36:41.653285+0000","last_unstale":"2024-05-08T06:36:43.109064+0000","last_undegraded":"2024-05-08T06:36:43.109064+0000","last_fullsized":"2024-05-08T06:36:43.109064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108884+0000","last_change":"2024-05-08T06:36:41.661193+0000","last_active":"2024-05-08T06:36:43.108884+0000","last_peered":"2024-05-08T06:36:43.108884+0000","last_clean":"2024-05-08T06:36:43.108884+0000","last_became_active":"2024-05-08T06:36:41.660907+0000","last_became_peered":"2024-05-08T06:36:41.660907+0000","last_unstale":"2024-05-08T06:36:43.108884+0000","last_undegraded":"2024-05-08T06:36:43.108884+0000","last_fullsized":"2024-05-08T06:36:43.108884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108493+0000","last_change":"2024-05-08T06:36:41.655350+0000","last_active":"2024-05-08T06:36:43.108493+0000","last_peered":"2024-05-08T06:36:43.108493+0000","last_clean":"2024-05-08T06:36:43.108493+0000","last_became_active":"2024-05-08T06:36:41.655064+0000","last_became_peered":"2024-05-08T06:36:41.655064+0000","last_unstale":"2024-05-08T06:36:43.108493+0000","last_undegraded":"2024-05-08T06:36:43.108493+0000","last_fullsized":"2024-05-08T06:36:43.108493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:11:54.263628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109972+0000","last_change":"2024-05-08T06:36:41.663210+0000","last_active":"2024-05-08T06:36:43.109972+0000","last_peered":"2024-05-08T06:36:43.109972+0000","last_clean":"2024-05-08T06:36:43.109972+0000","last_became_active":"2024-05-08T06:36:41.662954+0000","last_became_peered":"2024-05-08T06:36:41.662954+0000","last_unstale":"2024-05-08T06:36:43.109972+0000","last_undegraded":"2024-05-08T06:36:43.109972+0000","last_fullsized":"2024-05-08T06:36:43.109972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:36:32.652292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654330+0000","last_change":"2024-05-08T06:36:41.662910+0000","last_active":"2024-05-08T06:36:42.654330+0000","last_peered":"2024-05-08T06:36:42.654330+0000","last_clean":"2024-05-08T06:36:42.654330+0000","last_became_active":"2024-05-08T06:36:41.662585+0000","last_became_peered":"2024-05-08T06:36:41.662585+0000","last_unstale":"2024-05-08T06:36:42.654330+0000","last_undegraded":"2024-05-08T06:36:42.654330+0000","last_fullsized":"2024-05-08T06:36:42.654330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653737+0000","last_change":"2024-05-08T06:36:41.663863+0000","last_active":"2024-05-08T06:36:42.653737+0000","last_peered":"2024-05-08T06:36:42.653737+0000","last_clean":"2024-05-08T06:36:42.653737+0000","last_became_active":"2024-05-08T06:36:41.663584+0000","last_became_peered":"2024-05-08T06:36:41.663584+0000","last_unstale":"2024-05-08T06:36:42.653737+0000","last_undegraded":"2024-05-08T06:36:42.653737+0000","last_fullsized":"2024-05-08T06:36:42.653737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654617+0000","last_change":"2024-05-08T06:36:41.661979+0000","last_active":"2024-05-08T06:36:42.654617+0000","last_peered":"2024-05-08T06:36:42.654617+0000","last_clean":"2024-05-08T06:36:42.654617+0000","last_became_active":"2024-05-08T06:36:41.660222+0000","last_became_peered":"2024-05-08T06:36:41.660222+0000","last_unstale":"2024-05-08T06:36:42.654617+0000","last_undegraded":"2024-05-08T06:36:42.654617+0000","last_fullsized":"2024-05-08T06:36:42.654617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:26.842934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108871+0000","last_change":"2024-05-08T06:36:41.654832+0000","last_active":"2024-05-08T06:36:43.108871+0000","last_peered":"2024-05-08T06:36:43.108871+0000","last_clean":"2024-05-08T06:36:43.108871+0000","last_became_active":"2024-05-08T06:36:41.654550+0000","last_became_peered":"2024-05-08T06:36:41.654550+0000","last_unstale":"2024-05-08T06:36:43.108871+0000","last_undegraded":"2024-05-08T06:36:43.108871+0000","last_fullsized":"2024-05-08T06:36:43.108871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108766+0000","last_change":"2024-05-08T06:36:41.642429+0000","last_active":"2024-05-08T06:36:43.108766+0000","last_peered":"2024-05-08T06:36:43.108766+0000","last_clean":"2024-05-08T06:36:43.108766+0000","last_became_active":"2024-05-08T06:36:41.641956+0000","last_became_peered":"2024-05-08T06:36:41.641956+0000","last_unstale":"2024-05-08T06:36:43.108766+0000","last_undegraded":"2024-05-08T06:36:43.108766+0000","last_fullsized":"2024-05-08T06:36:43.108766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649200+0000","last_change":"2024-05-08T06:36:41.656423+0000","last_active":"2024-05-08T06:36:42.649200+0000","last_peered":"2024-05-08T06:36:42.649200+0000","last_clean":"2024-05-08T06:36:42.649200+0000","last_became_active":"2024-05-08T06:36:41.656154+0000","last_became_peered":"2024-05-08T06:36:41.656154+0000","last_unstale":"2024-05-08T06:36:42.649200+0000","last_undegraded":"2024-05-08T06:36:42.649200+0000","last_fullsized":"2024-05-08T06:36:42.649200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109681+0000","last_change":"2024-05-08T06:36:41.653378+0000","last_active":"2024-05-08T06:36:43.109681+0000","last_peered":"2024-05-08T06:36:43.109681+0000","last_clean":"2024-05-08T06:36:43.109681+0000","last_became_active":"2024-05-08T06:36:41.653027+0000","last_became_peered":"2024-05-08T06:36:41.653027+0000","last_unstale":"2024-05-08T06:36:43.109681+0000","last_undegraded":"2024-05-08T06:36:43.109681+0000","last_fullsized":"2024-05-08T06:36:43.109681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:02.654155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109919+0000","last_change":"2024-05-08T06:36:41.659055+0000","last_active":"2024-05-08T06:36:43.109919+0000","last_peered":"2024-05-08T06:36:43.109919+0000","last_clean":"2024-05-08T06:36:43.109919+0000","last_became_active":"2024-05-08T06:36:41.658774+0000","last_became_peered":"2024-05-08T06:36:41.658774+0000","last_unstale":"2024-05-08T06:36:43.109919+0000","last_undegraded":"2024-05-08T06:36:43.109919+0000","last_fullsized":"2024-05-08T06:36:43.109919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:38:12.052561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110931+0000","last_change":"2024-05-08T06:36:41.649669+0000","last_active":"2024-05-08T06:36:43.110931+0000","last_peered":"2024-05-08T06:36:43.110931+0000","last_clean":"2024-05-08T06:36:43.110931+0000","last_became_active":"2024-05-08T06:36:41.649391+0000","last_became_peered":"2024-05-08T06:36:41.649391+0000","last_unstale":"2024-05-08T06:36:43.110931+0000","last_undegraded":"2024-05-08T06:36:43.110931+0000","last_fullsized":"2024-05-08T06:36:43.110931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:04.579668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109443+0000","last_change":"2024-05-08T06:36:41.660852+0000","last_active":"2024-05-08T06:36:43.109443+0000","last_peered":"2024-05-08T06:36:43.109443+0000","last_clean":"2024-05-08T06:36:43.109443+0000","last_became_active":"2024-05-08T06:36:41.660573+0000","last_became_peered":"2024-05-08T06:36:41.660573+0000","last_unstale":"2024-05-08T06:36:43.109443+0000","last_undegraded":"2024-05-08T06:36:43.109443+0000","last_fullsized":"2024-05-08T06:36:43.109443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:02.654155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653881+0000","last_change":"2024-05-08T06:36:41.663211+0000","last_active":"2024-05-08T06:36:42.653881+0000","last_peered":"2024-05-08T06:36:42.653881+0000","last_clean":"2024-05-08T06:36:42.653881+0000","last_became_active":"2024-05-08T06:36:41.662936+0000","last_became_peered":"2024-05-08T06:36:41.662936+0000","last_unstale":"2024-05-08T06:36:42.653881+0000","last_undegraded":"2024-05-08T06:36:42.653881+0000","last_fullsized":"2024-05-08T06:36:42.653881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108430+0000","last_change":"2024-05-08T06:36:41.661453+0000","last_active":"2024-05-08T06:36:43.108430+0000","last_peered":"2024-05-08T06:36:43.108430+0000","last_clean":"2024-05-08T06:36:43.108430+0000","last_became_active":"2024-05-08T06:36:41.661146+0000","last_became_peered":"2024-05-08T06:36:41.661146+0000","last_unstale":"2024-05-08T06:36:43.108430+0000","last_undegraded":"2024-05-08T06:36:43.108430+0000","last_fullsized":"2024-05-08T06:36:43.108430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:11:54.263628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109377+0000","last_change":"2024-05-08T06:36:41.658664+0000","last_active":"2024-05-08T06:36:43.109377+0000","last_peered":"2024-05-08T06:36:43.109377+0000","last_clean":"2024-05-08T06:36:43.109377+0000","last_became_active":"2024-05-08T06:36:41.658352+0000","last_became_peered":"2024-05-08T06:36:41.658352+0000","last_unstale":"2024-05-08T06:36:43.109377+0000","last_undegraded":"2024-05-08T06:36:43.109377+0000","last_fullsized":"2024-05-08T06:36:43.109377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108694+0000","last_change":"2024-05-08T06:36:41.658967+0000","last_active":"2024-05-08T06:36:43.108694+0000","last_peered":"2024-05-08T06:36:43.108694+0000","last_clean":"2024-05-08T06:36:43.108694+0000","last_became_active":"2024-05-08T06:36:41.658387+0000","last_became_peered":"2024-05-08T06:36:41.658387+0000","last_unstale":"2024-05-08T06:36:43.108694+0000","last_undegraded":"2024-05-08T06:36:43.108694+0000","last_fullsized":"2024-05-08T06:36:43.108694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108892+0000","last_change":"2024-05-08T06:36:41.657102+0000","last_active":"2024-05-08T06:36:43.108892+0000","last_peered":"2024-05-08T06:36:43.108892+0000","last_clean":"2024-05-08T06:36:43.108892+0000","last_became_active":"2024-05-08T06:36:41.656844+0000","last_became_peered":"2024-05-08T06:36:41.656844+0000","last_unstale":"2024-05-08T06:36:43.108892+0000","last_undegraded":"2024-05-08T06:36:43.108892+0000","last_fullsized":"2024-05-08T06:36:43.108892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109150+0000","last_change":"2024-05-08T06:36:41.659592+0000","last_active":"2024-05-08T06:36:43.109150+0000","last_peered":"2024-05-08T06:36:43.109150+0000","last_clean":"2024-05-08T06:36:43.109150+0000","last_became_active":"2024-05-08T06:36:41.659295+0000","last_became_peered":"2024-05-08T06:36:41.659295+0000","last_unstale":"2024-05-08T06:36:43.109150+0000","last_undegraded":"2024-05-08T06:36:43.109150+0000","last_fullsized":"2024-05-08T06:36:43.109150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654479+0000","last_change":"2024-05-08T06:36:41.649572+0000","last_active":"2024-05-08T06:36:42.654479+0000","last_peered":"2024-05-08T06:36:42.654479+0000","last_clean":"2024-05-08T06:36:42.654479+0000","last_became_active":"2024-05-08T06:36:41.649302+0000","last_became_peered":"2024-05-08T06:36:41.649302+0000","last_unstale":"2024-05-08T06:36:42.654479+0000","last_undegraded":"2024-05-08T06:36:42.654479+0000","last_fullsized":"2024-05-08T06:36:42.654479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649400+0000","last_change":"2024-05-08T06:36:41.657182+0000","last_active":"2024-05-08T06:36:42.649400+0000","last_peered":"2024-05-08T06:36:42.649400+0000","last_clean":"2024-05-08T06:36:42.649400+0000","last_became_active":"2024-05-08T06:36:41.656919+0000","last_became_peered":"2024-05-08T06:36:41.656919+0000","last_unstale":"2024-05-08T06:36:42.649400+0000","last_undegraded":"2024-05-08T06:36:42.649400+0000","last_fullsized":"2024-05-08T06:36:42.649400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109121+0000","last_change":"2024-05-08T06:36:41.659147+0000","last_active":"2024-05-08T06:36:43.109121+0000","last_peered":"2024-05-08T06:36:43.109121+0000","last_clean":"2024-05-08T06:36:43.109121+0000","last_became_active":"2024-05-08T06:36:41.658880+0000","last_became_peered":"2024-05-08T06:36:41.658880+0000","last_unstale":"2024-05-08T06:36:43.109121+0000","last_undegraded":"2024-05-08T06:36:43.109121+0000","last_fullsized":"2024-05-08T06:36:43.109121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108909+0000","last_change":"2024-05-08T06:36:41.658925+0000","last_active":"2024-05-08T06:36:43.108909+0000","last_peered":"2024-05-08T06:36:43.108909+0000","last_clean":"2024-05-08T06:36:43.108909+0000","last_became_active":"2024-05-08T06:36:41.656879+0000","last_became_peered":"2024-05-08T06:36:41.656879+0000","last_unstale":"2024-05-08T06:36:43.108909+0000","last_undegraded":"2024-05-08T06:36:43.108909+0000","last_fullsized":"2024-05-08T06:36:43.108909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109551+0000","last_change":"2024-05-08T06:36:41.655125+0000","last_active":"2024-05-08T06:36:43.109551+0000","last_peered":"2024-05-08T06:36:43.109551+0000","last_clean":"2024-05-08T06:36:43.109551+0000","last_became_active":"2024-05-08T06:36:41.654817+0000","last_became_peered":"2024-05-08T06:36:41.654817+0000","last_unstale":"2024-05-08T06:36:43.109551+0000","last_undegraded":"2024-05-08T06:36:43.109551+0000","last_fullsized":"2024-05-08T06:36:43.109551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:42.444444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110786+0000","last_change":"2024-05-08T06:36:41.655361+0000","last_active":"2024-05-08T06:36:43.110786+0000","last_peered":"2024-05-08T06:36:43.110786+0000","last_clean":"2024-05-08T06:36:43.110786+0000","last_became_active":"2024-05-08T06:36:41.655085+0000","last_became_peered":"2024-05-08T06:36:41.655085+0000","last_unstale":"2024-05-08T06:36:43.110786+0000","last_undegraded":"2024-05-08T06:36:43.110786+0000","last_fullsized":"2024-05-08T06:36:43.110786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:26.842934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653569+0000","last_change":"2024-05-08T06:36:41.643791+0000","last_active":"2024-05-08T06:36:42.653569+0000","last_peered":"2024-05-08T06:36:42.653569+0000","last_clean":"2024-05-08T06:36:42.653569+0000","last_became_active":"2024-05-08T06:36:41.643506+0000","last_became_peered":"2024-05-08T06:36:41.643506+0000","last_unstale":"2024-05-08T06:36:42.653569+0000","last_undegraded":"2024-05-08T06:36:42.653569+0000","last_fullsized":"2024-05-08T06:36:42.653569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654375+0000","last_change":"2024-05-08T06:36:41.654106+0000","last_active":"2024-05-08T06:36:42.654375+0000","last_peered":"2024-05-08T06:36:42.654375+0000","last_clean":"2024-05-08T06:36:42.654375+0000","last_became_active":"2024-05-08T06:36:41.653822+0000","last_became_peered":"2024-05-08T06:36:41.653822+0000","last_unstale":"2024-05-08T06:36:42.654375+0000","last_undegraded":"2024-05-08T06:36:42.654375+0000","last_fullsized":"2024-05-08T06:36:42.654375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:51.358786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109209+0000","last_change":"2024-05-08T06:36:40.641349+0000","last_active":"2024-05-08T06:36:43.109209+0000","last_peered":"2024-05-08T06:36:43.109209+0000","last_clean":"2024-05-08T06:36:43.109209+0000","last_became_active":"2024-05-08T06:36:40.641086+0000","last_became_peered":"2024-05-08T06:36:40.641086+0000","last_unstale":"2024-05-08T06:36:43.109209+0000","last_undegraded":"2024-05-08T06:36:43.109209+0000","last_fullsized":"2024-05-08T06:36:43.109209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:39:09.616149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108579+0000","last_change":"2024-05-08T06:36:41.656120+0000","last_active":"2024-05-08T06:36:43.108579+0000","last_peered":"2024-05-08T06:36:43.108579+0000","last_clean":"2024-05-08T06:36:43.108579+0000","last_became_active":"2024-05-08T06:36:41.655843+0000","last_became_peered":"2024-05-08T06:36:41.655843+0000","last_unstale":"2024-05-08T06:36:43.108579+0000","last_undegraded":"2024-05-08T06:36:43.108579+0000","last_fullsized":"2024-05-08T06:36:43.108579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654066+0000","last_change":"2024-05-08T06:36:40.639740+0000","last_active":"2024-05-08T06:36:42.654066+0000","last_peered":"2024-05-08T06:36:42.654066+0000","last_clean":"2024-05-08T06:36:42.654066+0000","last_became_active":"2024-05-08T06:36:40.639370+0000","last_became_peered":"2024-05-08T06:36:40.639370+0000","last_unstale":"2024-05-08T06:36:42.654066+0000","last_undegraded":"2024-05-08T06:36:42.654066+0000","last_fullsized":"2024-05-08T06:36:42.654066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108684+0000","last_change":"2024-05-08T06:36:40.641983+0000","last_active":"2024-05-08T06:36:43.108684+0000","last_peered":"2024-05-08T06:36:43.108684+0000","last_clean":"2024-05-08T06:36:43.108684+0000","last_became_active":"2024-05-08T06:36:40.641666+0000","last_became_peered":"2024-05-08T06:36:40.641666+0000","last_unstale":"2024-05-08T06:36:43.108684+0000","last_undegraded":"2024-05-08T06:36:43.108684+0000","last_fullsized":"2024-05-08T06:36:43.108684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:47.997246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108516+0000","last_change":"2024-05-08T06:36:41.653257+0000","last_active":"2024-05-08T06:36:43.108516+0000","last_peered":"2024-05-08T06:36:43.108516+0000","last_clean":"2024-05-08T06:36:43.108516+0000","last_became_active":"2024-05-08T06:36:41.652900+0000","last_became_peered":"2024-05-08T06:36:41.652900+0000","last_unstale":"2024-05-08T06:36:43.108516+0000","last_undegraded":"2024-05-08T06:36:43.108516+0000","last_fullsized":"2024-05-08T06:36:43.108516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109272+0000","last_change":"2024-05-08T06:36:40.646856+0000","last_active":"2024-05-08T06:36:43.109272+0000","last_peered":"2024-05-08T06:36:43.109272+0000","last_clean":"2024-05-08T06:36:43.109272+0000","last_became_active":"2024-05-08T06:36:40.644640+0000","last_became_peered":"2024-05-08T06:36:40.644640+0000","last_unstale":"2024-05-08T06:36:43.109272+0000","last_undegraded":"2024-05-08T06:36:43.109272+0000","last_fullsized":"2024-05-08T06:36:43.109272+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649183+0000","last_change":"2024-05-08T06:36:41.656744+0000","last_active":"2024-05-08T06:36:42.649183+0000","last_peered":"2024-05-08T06:36:42.649183+0000","last_clean":"2024-05-08T06:36:42.649183+0000","last_became_active":"2024-05-08T06:36:41.656483+0000","last_became_peered":"2024-05-08T06:36:41.656483+0000","last_unstale":"2024-05-08T06:36:42.649183+0000","last_undegraded":"2024-05-08T06:36:42.649183+0000","last_fullsized":"2024-05-08T06:36:42.649183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653768+0000","last_change":"2024-05-08T06:36:40.648551+0000","last_active":"2024-05-08T06:36:42.653768+0000","last_peered":"2024-05-08T06:36:42.653768+0000","last_clean":"2024-05-08T06:36:42.653768+0000","last_became_active":"2024-05-08T06:36:40.648265+0000","last_became_peered":"2024-05-08T06:36:40.648265+0000","last_unstale":"2024-05-08T06:36:42.653768+0000","last_undegraded":"2024-05-08T06:36:42.653768+0000","last_fullsized":"2024-05-08T06:36:42.653768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109024+0000","last_change":"2024-05-08T06:36:41.658637+0000","last_active":"2024-05-08T06:36:43.109024+0000","last_peered":"2024-05-08T06:36:43.109024+0000","last_clean":"2024-05-08T06:36:43.109024+0000","last_became_active":"2024-05-08T06:36:41.658333+0000","last_became_peered":"2024-05-08T06:36:41.658333+0000","last_unstale":"2024-05-08T06:36:43.109024+0000","last_undegraded":"2024-05-08T06:36:43.109024+0000","last_fullsized":"2024-05-08T06:36:43.109024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109045+0000","last_change":"2024-05-08T06:36:40.647645+0000","last_active":"2024-05-08T06:36:43.109045+0000","last_peered":"2024-05-08T06:36:43.109045+0000","last_clean":"2024-05-08T06:36:43.109045+0000","last_became_active":"2024-05-08T06:36:40.647368+0000","last_became_peered":"2024-05-08T06:36:40.647368+0000","last_unstale":"2024-05-08T06:36:43.109045+0000","last_undegraded":"2024-05-08T06:36:43.109045+0000","last_fullsized":"2024-05-08T06:36:43.109045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109122+0000","last_change":"2024-05-08T06:36:41.662012+0000","last_active":"2024-05-08T06:36:43.109122+0000","last_peered":"2024-05-08T06:36:43.109122+0000","last_clean":"2024-05-08T06:36:43.109122+0000","last_became_active":"2024-05-08T06:36:41.661780+0000","last_became_peered":"2024-05-08T06:36:41.661780+0000","last_unstale":"2024-05-08T06:36:43.109122+0000","last_undegraded":"2024-05-08T06:36:43.109122+0000","last_fullsized":"2024-05-08T06:36:43.109122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649372+0000","last_change":"2024-05-08T06:36:40.649405+0000","last_active":"2024-05-08T06:36:42.649372+0000","last_peered":"2024-05-08T06:36:42.649372+0000","last_clean":"2024-05-08T06:36:42.649372+0000","last_became_active":"2024-05-08T06:36:40.649172+0000","last_became_peered":"2024-05-08T06:36:40.649172+0000","last_unstale":"2024-05-08T06:36:42.649372+0000","last_undegraded":"2024-05-08T06:36:42.649372+0000","last_fullsized":"2024-05-08T06:36:42.649372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654679+0000","last_change":"2024-05-08T06:36:41.654141+0000","last_active":"2024-05-08T06:36:42.654679+0000","last_peered":"2024-05-08T06:36:42.654679+0000","last_clean":"2024-05-08T06:36:42.654679+0000","last_became_active":"2024-05-08T06:36:41.653723+0000","last_became_peered":"2024-05-08T06:36:41.653723+0000","last_unstale":"2024-05-08T06:36:42.654679+0000","last_undegraded":"2024-05-08T06:36:42.654679+0000","last_fullsized":"2024-05-08T06:36:42.654679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:04.579668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654000+0000","last_change":"2024-05-08T06:36:40.641783+0000","last_active":"2024-05-08T06:36:42.654000+0000","last_peered":"2024-05-08T06:36:42.654000+0000","last_clean":"2024-05-08T06:36:42.654000+0000","last_became_active":"2024-05-08T06:36:40.641270+0000","last_became_peered":"2024-05-08T06:36:40.641270+0000","last_unstale":"2024-05-08T06:36:42.654000+0000","last_undegraded":"2024-05-08T06:36:42.654000+0000","last_fullsized":"2024-05-08T06:36:42.654000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108979+0000","last_change":"2024-05-08T06:36:41.657579+0000","last_active":"2024-05-08T06:36:43.108979+0000","last_peered":"2024-05-08T06:36:43.108979+0000","last_clean":"2024-05-08T06:36:43.108979+0000","last_became_active":"2024-05-08T06:36:41.657299+0000","last_became_peered":"2024-05-08T06:36:41.657299+0000","last_unstale":"2024-05-08T06:36:43.108979+0000","last_undegraded":"2024-05-08T06:36:43.108979+0000","last_fullsized":"2024-05-08T06:36:43.108979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:02.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108685+0000","last_change":"2024-05-08T06:36:40.641651+0000","last_active":"2024-05-08T06:36:43.108685+0000","last_peered":"2024-05-08T06:36:43.108685+0000","last_clean":"2024-05-08T06:36:43.108685+0000","last_became_active":"2024-05-08T06:36:40.641168+0000","last_became_peered":"2024-05-08T06:36:40.641168+0000","last_unstale":"2024-05-08T06:36:43.108685+0000","last_undegraded":"2024-05-08T06:36:43.108685+0000","last_fullsized":"2024-05-08T06:36:43.108685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:47.997246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108903+0000","last_change":"2024-05-08T06:36:41.661356+0000","last_active":"2024-05-08T06:36:43.108903+0000","last_peered":"2024-05-08T06:36:43.108903+0000","last_clean":"2024-05-08T06:36:43.108903+0000","last_became_active":"2024-05-08T06:36:41.661085+0000","last_became_peered":"2024-05-08T06:36:41.661085+0000","last_unstale":"2024-05-08T06:36:43.108903+0000","last_undegraded":"2024-05-08T06:36:43.108903+0000","last_fullsized":"2024-05-08T06:36:43.108903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109770+0000","last_change":"2024-05-08T06:36:40.646694+0000","last_active":"2024-05-08T06:36:43.109770+0000","last_peered":"2024-05-08T06:36:43.109770+0000","last_clean":"2024-05-08T06:36:43.109770+0000","last_became_active":"2024-05-08T06:36:40.645833+0000","last_became_peered":"2024-05-08T06:36:40.645833+0000","last_unstale":"2024-05-08T06:36:43.109770+0000","last_undegraded":"2024-05-08T06:36:43.109770+0000","last_fullsized":"2024-05-08T06:36:43.109770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109418+0000","last_change":"2024-05-08T06:36:41.668787+0000","last_active":"2024-05-08T06:36:43.109418+0000","last_peered":"2024-05-08T06:36:43.109418+0000","last_clean":"2024-05-08T06:36:43.109418+0000","last_became_active":"2024-05-08T06:36:41.668489+0000","last_became_peered":"2024-05-08T06:36:41.668489+0000","last_unstale":"2024-05-08T06:36:43.109418+0000","last_undegraded":"2024-05-08T06:36:43.109418+0000","last_fullsized":"2024-05-08T06:36:43.109418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.395237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653600+0000","last_change":"2024-05-08T06:36:40.640082+0000","last_active":"2024-05-08T06:36:42.653600+0000","last_peered":"2024-05-08T06:36:42.653600+0000","last_clean":"2024-05-08T06:36:42.653600+0000","last_became_active":"2024-05-08T06:36:40.638558+0000","last_became_peered":"2024-05-08T06:36:40.638558+0000","last_unstale":"2024-05-08T06:36:42.653600+0000","last_undegraded":"2024-05-08T06:36:42.653600+0000","last_fullsized":"2024-05-08T06:36:42.653600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653523+0000","last_change":"2024-05-08T06:36:41.643855+0000","last_active":"2024-05-08T06:36:42.653523+0000","last_peered":"2024-05-08T06:36:42.653523+0000","last_clean":"2024-05-08T06:36:42.653523+0000","last_became_active":"2024-05-08T06:36:41.643117+0000","last_became_peered":"2024-05-08T06:36:41.643117+0000","last_unstale":"2024-05-08T06:36:42.653523+0000","last_undegraded":"2024-05-08T06:36:42.653523+0000","last_fullsized":"2024-05-08T06:36:42.653523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110713+0000","last_change":"2024-05-08T06:36:40.642303+0000","last_active":"2024-05-08T06:36:43.110713+0000","last_peered":"2024-05-08T06:36:43.110713+0000","last_clean":"2024-05-08T06:36:43.110713+0000","last_became_active":"2024-05-08T06:36:40.642042+0000","last_became_peered":"2024-05-08T06:36:40.642042+0000","last_unstale":"2024-05-08T06:36:43.110713+0000","last_undegraded":"2024-05-08T06:36:43.110713+0000","last_fullsized":"2024-05-08T06:36:43.110713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109058+0000","last_change":"2024-05-08T06:36:41.660559+0000","last_active":"2024-05-08T06:36:43.109058+0000","last_peered":"2024-05-08T06:36:43.109058+0000","last_clean":"2024-05-08T06:36:43.109058+0000","last_became_active":"2024-05-08T06:36:41.660282+0000","last_became_peered":"2024-05-08T06:36:41.660282+0000","last_unstale":"2024-05-08T06:36:43.109058+0000","last_undegraded":"2024-05-08T06:36:43.109058+0000","last_fullsized":"2024-05-08T06:36:43.109058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:49.740862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654374+0000","last_change":"2024-05-08T06:36:40.646655+0000","last_active":"2024-05-08T06:36:42.654374+0000","last_peered":"2024-05-08T06:36:42.654374+0000","last_clean":"2024-05-08T06:36:42.654374+0000","last_became_active":"2024-05-08T06:36:40.646346+0000","last_became_peered":"2024-05-08T06:36:40.646346+0000","last_unstale":"2024-05-08T06:36:42.654374+0000","last_undegraded":"2024-05-08T06:36:42.654374+0000","last_fullsized":"2024-05-08T06:36:42.654374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109890+0000","last_change":"2024-05-08T06:36:41.659817+0000","last_active":"2024-05-08T06:36:43.109890+0000","last_peered":"2024-05-08T06:36:43.109890+0000","last_clean":"2024-05-08T06:36:43.109890+0000","last_became_active":"2024-05-08T06:36:41.659524+0000","last_became_peered":"2024-05-08T06:36:41.659524+0000","last_unstale":"2024-05-08T06:36:43.109890+0000","last_undegraded":"2024-05-08T06:36:43.109890+0000","last_fullsized":"2024-05-08T06:36:43.109890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:38:12.052561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110881+0000","last_change":"2024-05-08T06:36:40.641491+0000","last_active":"2024-05-08T06:36:43.110881+0000","last_peered":"2024-05-08T06:36:43.110881+0000","last_clean":"2024-05-08T06:36:43.110881+0000","last_became_active":"2024-05-08T06:36:40.641207+0000","last_became_peered":"2024-05-08T06:36:40.641207+0000","last_unstale":"2024-05-08T06:36:43.110881+0000","last_undegraded":"2024-05-08T06:36:43.110881+0000","last_fullsized":"2024-05-08T06:36:43.110881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110049+0000","last_change":"2024-05-08T06:36:41.667992+0000","last_active":"2024-05-08T06:36:43.110049+0000","last_peered":"2024-05-08T06:36:43.110049+0000","last_clean":"2024-05-08T06:36:43.110049+0000","last_became_active":"2024-05-08T06:36:41.667733+0000","last_became_peered":"2024-05-08T06:36:41.667733+0000","last_unstale":"2024-05-08T06:36:43.110049+0000","last_undegraded":"2024-05-08T06:36:43.110049+0000","last_fullsized":"2024-05-08T06:36:43.110049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:41:00.520597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649448+0000","last_change":"2024-05-08T06:36:40.646069+0000","last_active":"2024-05-08T06:36:42.649448+0000","last_peered":"2024-05-08T06:36:42.649448+0000","last_clean":"2024-05-08T06:36:42.649448+0000","last_became_active":"2024-05-08T06:36:40.645769+0000","last_became_peered":"2024-05-08T06:36:40.645769+0000","last_unstale":"2024-05-08T06:36:42.649448+0000","last_undegraded":"2024-05-08T06:36:42.649448+0000","last_fullsized":"2024-05-08T06:36:42.649448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:25.837342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109628+0000","last_change":"2024-05-08T06:36:41.660346+0000","last_active":"2024-05-08T06:36:43.109628+0000","last_peered":"2024-05-08T06:36:43.109628+0000","last_clean":"2024-05-08T06:36:43.109628+0000","last_became_active":"2024-05-08T06:36:41.660028+0000","last_became_peered":"2024-05-08T06:36:41.660028+0000","last_unstale":"2024-05-08T06:36:43.109628+0000","last_undegraded":"2024-05-08T06:36:43.109628+0000","last_fullsized":"2024-05-08T06:36:43.109628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:14.886723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110847+0000","last_change":"2024-05-08T06:36:40.647112+0000","last_active":"2024-05-08T06:36:43.110847+0000","last_peered":"2024-05-08T06:36:43.110847+0000","last_clean":"2024-05-08T06:36:43.110847+0000","last_became_active":"2024-05-08T06:36:40.646804+0000","last_became_peered":"2024-05-08T06:36:40.646804+0000","last_unstale":"2024-05-08T06:36:43.110847+0000","last_undegraded":"2024-05-08T06:36:43.110847+0000","last_fullsized":"2024-05-08T06:36:43.110847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109148+0000","last_change":"2024-05-08T06:36:41.663875+0000","last_active":"2024-05-08T06:36:43.109148+0000","last_peered":"2024-05-08T06:36:43.109148+0000","last_clean":"2024-05-08T06:36:43.109148+0000","last_became_active":"2024-05-08T06:36:41.663595+0000","last_became_peered":"2024-05-08T06:36:41.663595+0000","last_unstale":"2024-05-08T06:36:43.109148+0000","last_undegraded":"2024-05-08T06:36:43.109148+0000","last_fullsized":"2024-05-08T06:36:43.109148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:42.444444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108725+0000","last_change":"2024-05-08T06:36:31.710287+0000","last_active":"2024-05-08T06:36:43.108725+0000","last_peered":"2024-05-08T06:36:43.108725+0000","last_clean":"2024-05-08T06:36:43.108725+0000","last_became_active":"2024-05-08T06:36:31.709270+0000","last_became_peered":"2024-05-08T06:36:31.709270+0000","last_unstale":"2024-05-08T06:36:43.108725+0000","last_undegraded":"2024-05-08T06:36:43.108725+0000","last_fullsized":"2024-05-08T06:36:43.108725+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_clean_scrub_stamp":"2024-05-08T06:36:30.122918+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:42.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108783+0000","last_change":"2024-05-08T06:36:40.645536+0000","last_active":"2024-05-08T06:36:43.108783+0000","last_peered":"2024-05-08T06:36:43.108783+0000","last_clean":"2024-05-08T06:36:43.108783+0000","last_became_active":"2024-05-08T06:36:40.644256+0000","last_became_peered":"2024-05-08T06:36:40.644256+0000","last_unstale":"2024-05-08T06:36:43.108783+0000","last_undegraded":"2024-05-08T06:36:43.108783+0000","last_fullsized":"2024-05-08T06:36:43.108783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109026+0000","last_change":"2024-05-08T06:36:41.655301+0000","last_active":"2024-05-08T06:36:43.109026+0000","last_peered":"2024-05-08T06:36:43.109026+0000","last_clean":"2024-05-08T06:36:43.109026+0000","last_became_active":"2024-05-08T06:36:41.655090+0000","last_became_peered":"2024-05-08T06:36:41.655090+0000","last_unstale":"2024-05-08T06:36:43.109026+0000","last_undegraded":"2024-05-08T06:36:43.109026+0000","last_fullsized":"2024-05-08T06:36:43.109026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108653+0000","last_change":"2024-05-08T06:36:40.647913+0000","last_active":"2024-05-08T06:36:43.108653+0000","last_peered":"2024-05-08T06:36:43.108653+0000","last_clean":"2024-05-08T06:36:43.108653+0000","last_became_active":"2024-05-08T06:36:40.647638+0000","last_became_peered":"2024-05-08T06:36:40.647638+0000","last_unstale":"2024-05-08T06:36:43.108653+0000","last_undegraded":"2024-05-08T06:36:43.108653+0000","last_fullsized":"2024-05-08T06:36:43.108653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:29:44.646218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108285+0000","last_change":"2024-05-08T06:36:41.658563+0000","last_active":"2024-05-08T06:36:43.108285+0000","last_peered":"2024-05-08T06:36:43.108285+0000","last_clean":"2024-05-08T06:36:43.108285+0000","last_became_active":"2024-05-08T06:36:41.657157+0000","last_became_peered":"2024-05-08T06:36:41.657157+0000","last_unstale":"2024-05-08T06:36:43.108285+0000","last_undegraded":"2024-05-08T06:36:43.108285+0000","last_fullsized":"2024-05-08T06:36:43.108285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:30:14.798179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110812+0000","last_change":"2024-05-08T06:36:40.642187+0000","last_active":"2024-05-08T06:36:43.110812+0000","last_peered":"2024-05-08T06:36:43.110812+0000","last_clean":"2024-05-08T06:36:43.110812+0000","last_became_active":"2024-05-08T06:36:40.641906+0000","last_became_peered":"2024-05-08T06:36:40.641906+0000","last_unstale":"2024-05-08T06:36:43.110812+0000","last_undegraded":"2024-05-08T06:36:43.110812+0000","last_fullsized":"2024-05-08T06:36:43.110812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649176+0000","last_change":"2024-05-08T06:36:41.642328+0000","last_active":"2024-05-08T06:36:42.649176+0000","last_peered":"2024-05-08T06:36:42.649176+0000","last_clean":"2024-05-08T06:36:42.649176+0000","last_became_active":"2024-05-08T06:36:41.642053+0000","last_became_peered":"2024-05-08T06:36:41.642053+0000","last_unstale":"2024-05-08T06:36:42.649176+0000","last_undegraded":"2024-05-08T06:36:42.649176+0000","last_fullsized":"2024-05-08T06:36:42.649176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109887+0000","last_change":"2024-05-08T06:36:40.646469+0000","last_active":"2024-05-08T06:36:43.109887+0000","last_peered":"2024-05-08T06:36:43.109887+0000","last_clean":"2024-05-08T06:36:43.109887+0000","last_became_active":"2024-05-08T06:36:40.646207+0000","last_became_peered":"2024-05-08T06:36:40.646207+0000","last_unstale":"2024-05-08T06:36:43.109887+0000","last_undegraded":"2024-05-08T06:36:43.109887+0000","last_fullsized":"2024-05-08T06:36:43.109887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654706+0000","last_change":"2024-05-08T06:36:41.654729+0000","last_active":"2024-05-08T06:36:42.654706+0000","last_peered":"2024-05-08T06:36:42.654706+0000","last_clean":"2024-05-08T06:36:42.654706+0000","last_became_active":"2024-05-08T06:36:41.654466+0000","last_became_peered":"2024-05-08T06:36:41.654466+0000","last_unstale":"2024-05-08T06:36:42.654706+0000","last_undegraded":"2024-05-08T06:36:42.654706+0000","last_fullsized":"2024-05-08T06:36:42.654706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:44.798191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649223+0000","last_change":"2024-05-08T06:36:40.647304+0000","last_active":"2024-05-08T06:36:42.649223+0000","last_peered":"2024-05-08T06:36:42.649223+0000","last_clean":"2024-05-08T06:36:42.649223+0000","last_became_active":"2024-05-08T06:36:40.647010+0000","last_became_peered":"2024-05-08T06:36:40.647010+0000","last_unstale":"2024-05-08T06:36:42.649223+0000","last_undegraded":"2024-05-08T06:36:42.649223+0000","last_fullsized":"2024-05-08T06:36:42.649223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:50.353194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109655+0000","last_change":"2024-05-08T06:36:41.656064+0000","last_active":"2024-05-08T06:36:43.109655+0000","last_peered":"2024-05-08T06:36:43.109655+0000","last_clean":"2024-05-08T06:36:43.109655+0000","last_became_active":"2024-05-08T06:36:41.655714+0000","last_became_peered":"2024-05-08T06:36:41.655714+0000","last_unstale":"2024-05-08T06:36:43.109655+0000","last_undegraded":"2024-05-08T06:36:43.109655+0000","last_fullsized":"2024-05-08T06:36:43.109655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.395237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109737+0000","last_change":"2024-05-08T06:36:40.648662+0000","last_active":"2024-05-08T06:36:43.109737+0000","last_peered":"2024-05-08T06:36:43.109737+0000","last_clean":"2024-05-08T06:36:43.109737+0000","last_became_active":"2024-05-08T06:36:40.648367+0000","last_became_peered":"2024-05-08T06:36:40.648367+0000","last_unstale":"2024-05-08T06:36:43.109737+0000","last_undegraded":"2024-05-08T06:36:43.109737+0000","last_fullsized":"2024-05-08T06:36:43.109737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:25.837342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649254+0000","last_change":"2024-05-08T06:36:41.642330+0000","last_active":"2024-05-08T06:36:42.649254+0000","last_peered":"2024-05-08T06:36:42.649254+0000","last_clean":"2024-05-08T06:36:42.649254+0000","last_became_active":"2024-05-08T06:36:41.642059+0000","last_became_peered":"2024-05-08T06:36:41.642059+0000","last_unstale":"2024-05-08T06:36:42.649254+0000","last_undegraded":"2024-05-08T06:36:42.649254+0000","last_fullsized":"2024-05-08T06:36:42.649254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649350+0000","last_change":"2024-05-08T06:36:40.646675+0000","last_active":"2024-05-08T06:36:42.649350+0000","last_peered":"2024-05-08T06:36:42.649350+0000","last_clean":"2024-05-08T06:36:42.649350+0000","last_became_active":"2024-05-08T06:36:40.646346+0000","last_became_peered":"2024-05-08T06:36:40.646346+0000","last_unstale":"2024-05-08T06:36:42.649350+0000","last_undegraded":"2024-05-08T06:36:42.649350+0000","last_fullsized":"2024-05-08T06:36:42.649350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:03.574076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109189+0000","last_change":"2024-05-08T06:36:41.658633+0000","last_active":"2024-05-08T06:36:43.109189+0000","last_peered":"2024-05-08T06:36:43.109189+0000","last_clean":"2024-05-08T06:36:43.109189+0000","last_became_active":"2024-05-08T06:36:41.658294+0000","last_became_peered":"2024-05-08T06:36:41.658294+0000","last_unstale":"2024-05-08T06:36:43.109189+0000","last_undegraded":"2024-05-08T06:36:43.109189+0000","last_fullsized":"2024-05-08T06:36:43.109189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653882+0000","last_change":"2024-05-08T06:36:40.639547+0000","last_active":"2024-05-08T06:36:42.653882+0000","last_peered":"2024-05-08T06:36:42.653882+0000","last_clean":"2024-05-08T06:36:42.653882+0000","last_became_active":"2024-05-08T06:36:40.639240+0000","last_became_peered":"2024-05-08T06:36:40.639240+0000","last_unstale":"2024-05-08T06:36:42.653882+0000","last_undegraded":"2024-05-08T06:36:42.653882+0000","last_fullsized":"2024-05-08T06:36:42.653882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108530+0000","last_change":"2024-05-08T06:36:41.648709+0000","last_active":"2024-05-08T06:36:43.108530+0000","last_peered":"2024-05-08T06:36:43.108530+0000","last_clean":"2024-05-08T06:36:43.108530+0000","last_became_active":"2024-05-08T06:36:41.648406+0000","last_became_peered":"2024-05-08T06:36:41.648406+0000","last_unstale":"2024-05-08T06:36:43.108530+0000","last_undegraded":"2024-05-08T06:36:43.108530+0000","last_fullsized":"2024-05-08T06:36:43.108530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108653+0000","last_change":"2024-05-08T06:36:40.648721+0000","last_active":"2024-05-08T06:36:43.108653+0000","last_peered":"2024-05-08T06:36:43.108653+0000","last_clean":"2024-05-08T06:36:43.108653+0000","last_became_active":"2024-05-08T06:36:40.648429+0000","last_became_peered":"2024-05-08T06:36:40.648429+0000","last_unstale":"2024-05-08T06:36:43.108653+0000","last_undegraded":"2024-05-08T06:36:43.108653+0000","last_fullsized":"2024-05-08T06:36:43.108653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:43.792599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654290+0000","last_change":"2024-05-08T06:36:41.655289+0000","last_active":"2024-05-08T06:36:42.654290+0000","last_peered":"2024-05-08T06:36:42.654290+0000","last_clean":"2024-05-08T06:36:42.654290+0000","last_became_active":"2024-05-08T06:36:41.655022+0000","last_became_peered":"2024-05-08T06:36:41.655022+0000","last_unstale":"2024-05-08T06:36:42.654290+0000","last_undegraded":"2024-05-08T06:36:42.654290+0000","last_fullsized":"2024-05-08T06:36:42.654290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:57.405934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649256+0000","last_change":"2024-05-08T06:36:40.645335+0000","last_active":"2024-05-08T06:36:42.649256+0000","last_peered":"2024-05-08T06:36:42.649256+0000","last_clean":"2024-05-08T06:36:42.649256+0000","last_became_active":"2024-05-08T06:36:40.645019+0000","last_became_peered":"2024-05-08T06:36:40.645019+0000","last_unstale":"2024-05-08T06:36:42.649256+0000","last_undegraded":"2024-05-08T06:36:42.649256+0000","last_fullsized":"2024-05-08T06:36:42.649256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108851+0000","last_change":"2024-05-08T06:36:41.653910+0000","last_active":"2024-05-08T06:36:43.108851+0000","last_peered":"2024-05-08T06:36:43.108851+0000","last_clean":"2024-05-08T06:36:43.108851+0000","last_became_active":"2024-05-08T06:36:41.653567+0000","last_became_peered":"2024-05-08T06:36:41.653567+0000","last_unstale":"2024-05-08T06:36:43.108851+0000","last_undegraded":"2024-05-08T06:36:43.108851+0000","last_fullsized":"2024-05-08T06:36:43.108851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649426+0000","last_change":"2024-05-08T06:36:40.642880+0000","last_active":"2024-05-08T06:36:42.649426+0000","last_peered":"2024-05-08T06:36:42.649426+0000","last_clean":"2024-05-08T06:36:42.649426+0000","last_became_active":"2024-05-08T06:36:40.642605+0000","last_became_peered":"2024-05-08T06:36:40.642605+0000","last_unstale":"2024-05-08T06:36:42.649426+0000","last_undegraded":"2024-05-08T06:36:42.649426+0000","last_fullsized":"2024-05-08T06:36:42.649426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109118+0000","last_change":"2024-05-08T06:36:41.654124+0000","last_active":"2024-05-08T06:36:43.109118+0000","last_peered":"2024-05-08T06:36:43.109118+0000","last_clean":"2024-05-08T06:36:43.109118+0000","last_became_active":"2024-05-08T06:36:41.653837+0000","last_became_peered":"2024-05-08T06:36:41.653837+0000","last_unstale":"2024-05-08T06:36:43.109118+0000","last_undegraded":"2024-05-08T06:36:43.109118+0000","last_fullsized":"2024-05-08T06:36:43.109118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109611+0000","last_change":"2024-05-08T06:36:40.644938+0000","last_active":"2024-05-08T06:36:43.109611+0000","last_peered":"2024-05-08T06:36:43.109611+0000","last_clean":"2024-05-08T06:36:43.109611+0000","last_became_active":"2024-05-08T06:36:40.644549+0000","last_became_peered":"2024-05-08T06:36:40.644549+0000","last_unstale":"2024-05-08T06:36:43.109611+0000","last_undegraded":"2024-05-08T06:36:43.109611+0000","last_fullsized":"2024-05-08T06:36:43.109611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:50.353194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653954+0000","last_change":"2024-05-08T06:36:41.659401+0000","last_active":"2024-05-08T06:36:42.653954+0000","last_peered":"2024-05-08T06:36:42.653954+0000","last_clean":"2024-05-08T06:36:42.653954+0000","last_became_active":"2024-05-08T06:36:41.659063+0000","last_became_peered":"2024-05-08T06:36:41.659063+0000","last_unstale":"2024-05-08T06:36:42.653954+0000","last_undegraded":"2024-05-08T06:36:42.653954+0000","last_fullsized":"2024-05-08T06:36:42.653954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109449+0000","last_change":"2024-05-08T06:36:41.655858+0000","last_active":"2024-05-08T06:36:43.109449+0000","last_peered":"2024-05-08T06:36:43.109449+0000","last_clean":"2024-05-08T06:36:43.109449+0000","last_became_active":"2024-05-08T06:36:41.655569+0000","last_became_peered":"2024-05-08T06:36:41.655569+0000","last_unstale":"2024-05-08T06:36:43.109449+0000","last_undegraded":"2024-05-08T06:36:43.109449+0000","last_fullsized":"2024-05-08T06:36:43.109449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:02.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109476+0000","last_change":"2024-05-08T06:36:40.646643+0000","last_active":"2024-05-08T06:36:43.109476+0000","last_peered":"2024-05-08T06:36:43.109476+0000","last_clean":"2024-05-08T06:36:43.109476+0000","last_became_active":"2024-05-08T06:36:40.646319+0000","last_became_peered":"2024-05-08T06:36:40.646319+0000","last_unstale":"2024-05-08T06:36:43.109476+0000","last_undegraded":"2024-05-08T06:36:43.109476+0000","last_fullsized":"2024-05-08T06:36:43.109476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:03.574076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109441+0000","last_change":"2024-05-08T06:36:41.656383+0000","last_active":"2024-05-08T06:36:43.109441+0000","last_peered":"2024-05-08T06:36:43.109441+0000","last_clean":"2024-05-08T06:36:43.109441+0000","last_became_active":"2024-05-08T06:36:41.656059+0000","last_became_peered":"2024-05-08T06:36:41.656059+0000","last_unstale":"2024-05-08T06:36:43.109441+0000","last_undegraded":"2024-05-08T06:36:43.109441+0000","last_fullsized":"2024-05-08T06:36:43.109441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:49.740862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109265+0000","last_change":"2024-05-08T06:36:40.646560+0000","last_active":"2024-05-08T06:36:43.109265+0000","last_peered":"2024-05-08T06:36:43.109265+0000","last_clean":"2024-05-08T06:36:43.109265+0000","last_became_active":"2024-05-08T06:36:40.646285+0000","last_became_peered":"2024-05-08T06:36:40.646285+0000","last_unstale":"2024-05-08T06:36:43.109265+0000","last_undegraded":"2024-05-08T06:36:43.109265+0000","last_fullsized":"2024-05-08T06:36:43.109265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108439+0000","last_change":"2024-05-08T06:36:41.661975+0000","last_active":"2024-05-08T06:36:43.108439+0000","last_peered":"2024-05-08T06:36:43.108439+0000","last_clean":"2024-05-08T06:36:43.108439+0000","last_became_active":"2024-05-08T06:36:41.661759+0000","last_became_peered":"2024-05-08T06:36:41.661759+0000","last_unstale":"2024-05-08T06:36:43.108439+0000","last_undegraded":"2024-05-08T06:36:43.108439+0000","last_fullsized":"2024-05-08T06:36:43.108439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109215+0000","last_change":"2024-05-08T06:36:40.645426+0000","last_active":"2024-05-08T06:36:43.109215+0000","last_peered":"2024-05-08T06:36:43.109215+0000","last_clean":"2024-05-08T06:36:43.109215+0000","last_became_active":"2024-05-08T06:36:40.645194+0000","last_became_peered":"2024-05-08T06:36:40.645194+0000","last_unstale":"2024-05-08T06:36:43.109215+0000","last_undegraded":"2024-05-08T06:36:43.109215+0000","last_fullsized":"2024-05-08T06:36:43.109215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109153+0000","last_change":"2024-05-08T06:36:41.660148+0000","last_active":"2024-05-08T06:36:43.109153+0000","last_peered":"2024-05-08T06:36:43.109153+0000","last_clean":"2024-05-08T06:36:43.109153+0000","last_became_active":"2024-05-08T06:36:41.659907+0000","last_became_peered":"2024-05-08T06:36:41.659907+0000","last_unstale":"2024-05-08T06:36:43.109153+0000","last_undegraded":"2024-05-08T06:36:43.109153+0000","last_fullsized":"2024-05-08T06:36:43.109153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:45:07.961465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109158+0000","last_change":"2024-05-08T06:36:40.645379+0000","last_active":"2024-05-08T06:36:43.109158+0000","last_peered":"2024-05-08T06:36:43.109158+0000","last_clean":"2024-05-08T06:36:43.109158+0000","last_became_active":"2024-05-08T06:36:40.645091+0000","last_became_peered":"2024-05-08T06:36:40.645091+0000","last_unstale":"2024-05-08T06:36:43.109158+0000","last_undegraded":"2024-05-08T06:36:43.109158+0000","last_fullsized":"2024-05-08T06:36:43.109158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:36:54.504 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:36:54.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.640+0000 7f6f379da640 1 Processor -- start 2024-05-08T06:36:54.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.640+0000 7f6f379da640 1 -- start start 2024-05-08T06:36:54.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f379da640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6f3015f4c0 0x7f6f300d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f379da640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6f300baf20 0x7f6f300d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f379da640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f300b9d10 con 0x7f6f300d3820 2024-05-08T06:36:54.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f379da640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f6f300d3390 con 0x7f6f3015f4c0 2024-05-08T06:36:54.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f379da640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6f300b9ff0 con 0x7f6f300baf20 2024-05-08T06:36:54.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f35776640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6f3015f4c0 0x7f6f300d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f35776640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6f3015f4c0 0x7f6f300d21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44778/0 (socket says 172.21.15.163:44778) 2024-05-08T06:36:54.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f35f77640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6f300d3820 0x7f6f300d3cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57194/0 (socket says 172.21.15.163:57194) 2024-05-08T06:36:54.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f35776640 1 -- 172.21.15.163:0/563616038 learned_addr learned my addr 172.21.15.163:0/563616038 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:54.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f34f75640 1 --2- 172.21.15.163:0/563616038 >> v2:172.21.15.192:3300/0 conn(0x7f6f300baf20 0x7f6f300d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f35776640 1 -- 172.21.15.163:0/563616038 >> v2:172.21.15.192:3300/0 conn(0x7f6f300baf20 msgr2=0x7f6f300d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:54.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f35776640 1 --2- 172.21.15.163:0/563616038 >> v2:172.21.15.192:3300/0 conn(0x7f6f300baf20 0x7f6f300d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f35776640 1 -- 172.21.15.163:0/563616038 >> v1:172.21.15.163:6789/0 conn(0x7f6f300d3820 legacy=0x7f6f300d3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f35776640 1 -- 172.21.15.163:0/563616038 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f300ba2d0 con 0x7f6f3015f4c0 2024-05-08T06:36:54.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.641+0000 7f6f27fff640 1 -- 172.21.15.163:0/563616038 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3711297989 0 0) 0x7f6f300b9d10 con 0x7f6f300d3820 2024-05-08T06:36:54.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.642+0000 7f6f35776640 1 --2- 172.21.15.163:0/563616038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f3015f4c0 0x7f6f300d21b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6f300b74a0 tx=0x7f6f2c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630e81a98e8ce559 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:54.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.642+0000 7f6f27fff640 1 -- 172.21.15.163:0/563616038 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c037070 con 0x7f6f3015f4c0 2024-05-08T06:36:54.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.642+0000 7f6f27fff640 1 -- 172.21.15.163:0/563616038 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2c02ae50 con 0x7f6f3015f4c0 2024-05-08T06:36:54.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.642+0000 7f6f27fff640 1 -- 172.21.15.163:0/563616038 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c032860 con 0x7f6f3015f4c0 2024-05-08T06:36:54.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.642+0000 7f6f379da640 1 -- 172.21.15.163:0/563616038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f3015f4c0 msgr2=0x7f6f300d21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.642+0000 7f6f379da640 1 --2- 172.21.15.163:0/563616038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f3015f4c0 0x7f6f300d21b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6f300b74a0 tx=0x7f6f2c02a300 comp rx=0 tx=0).stop 2024-05-08T06:36:54.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.643+0000 7f6f379da640 1 -- 172.21.15.163:0/563616038 shutdown_connections 2024-05-08T06:36:54.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.643+0000 7f6f379da640 1 --2- 172.21.15.163:0/563616038 >> v2:172.21.15.192:3300/0 conn(0x7f6f300baf20 0x7f6f300d26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.643+0000 7f6f379da640 1 --2- 172.21.15.163:0/563616038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f3015f4c0 0x7f6f300d21b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.643 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.643+0000 7f6f379da640 1 -- 172.21.15.163:0/563616038 >> 172.21.15.163:0/563616038 conn(0x7f6f30073e70 msgr2=0x7f6f30074280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:54.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.643+0000 7f6f379da640 1 -- 172.21.15.163:0/563616038 shutdown_connections 2024-05-08T06:36:54.644 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.643+0000 7f6f379da640 1 -- 172.21.15.163:0/563616038 wait complete. 2024-05-08T06:36:54.646 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.644+0000 7f6f379da640 1 Processor -- start 2024-05-08T06:36:54.646 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.644+0000 7f6f379da640 1 -- start start 2024-05-08T06:36:54.646 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f379da640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f300baf20 0x7f6f301f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.646 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f379da640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f300d3820 0x7f6f300c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f379da640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f3015f4c0 0x7f6f30160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f379da640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6f300d4540 con 0x7f6f3015f4c0 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f379da640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6f300d43c0 con 0x7f6f300baf20 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f379da640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6f300d46c0 con 0x7f6f300d3820 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f35f77640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f3015f4c0 0x7f6f30160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f35f77640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f3015f4c0 0x7f6f30160320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44786/0 (socket says 172.21.15.163:44786) 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f35f77640 1 -- 172.21.15.163:0/877899004 learned_addr learned my addr 172.21.15.163:0/877899004 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f35776640 1 --2- 172.21.15.163:0/877899004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f300baf20 0x7f6f301f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f34f75640 1 --2- 172.21.15.163:0/877899004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f300d3820 0x7f6f300c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f35f77640 1 -- 172.21.15.163:0/877899004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f300d3820 msgr2=0x7f6f300c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f35f77640 1 --2- 172.21.15.163:0/877899004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f300d3820 0x7f6f300c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f35f77640 1 -- 172.21.15.163:0/877899004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f300baf20 msgr2=0x7f6f301f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f35f77640 1 --2- 172.21.15.163:0/877899004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f300baf20 0x7f6f301f4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f35f77640 1 -- 172.21.15.163:0/877899004 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f30160860 con 0x7f6f3015f4c0 2024-05-08T06:36:54.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f35f77640 1 --2- 172.21.15.163:0/877899004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f3015f4c0 0x7f6f30160320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800c7a0 tx=0x7f6f2800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:54.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f25ffb640 1 -- 172.21.15.163:0/877899004 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28018070 con 0x7f6f3015f4c0 2024-05-08T06:36:54.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f25ffb640 1 -- 172.21.15.163:0/877899004 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2800a5e0 con 0x7f6f3015f4c0 2024-05-08T06:36:54.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f379da640 1 -- 172.21.15.163:0/877899004 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f300c9ed0 con 0x7f6f3015f4c0 2024-05-08T06:36:54.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f25ffb640 1 -- 172.21.15.163:0/877899004 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2800aa50 con 0x7f6f3015f4c0 2024-05-08T06:36:54.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.645+0000 7f6f379da640 1 -- 172.21.15.163:0/877899004 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f301615c0 con 0x7f6f3015f4c0 2024-05-08T06:36:54.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.647+0000 7f6f379da640 1 -- 172.21.15.163:0/877899004 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f3006a550 con 0x7f6f3015f4c0 2024-05-08T06:36:54.648 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.647+0000 7f6f25ffb640 1 -- 172.21.15.163:0/877899004 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6f2801b070 con 0x7f6f3015f4c0 2024-05-08T06:36:54.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.648+0000 7f6f25ffb640 1 --2- 172.21.15.163:0/877899004 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6f140762d0 0x7f6f14078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.648+0000 7f6f35776640 1 --2- 172.21.15.163:0/877899004 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6f140762d0 0x7f6f14078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.648+0000 7f6f25ffb640 1 -- 172.21.15.163:0/877899004 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6f28096b80 con 0x7f6f3015f4c0 2024-05-08T06:36:54.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.648+0000 7f6f35776640 1 --2- 172.21.15.163:0/877899004 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6f140762d0 0x7f6f14078790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c034000 tx=0x7f6f2c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:54.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.652+0000 7f6f25ffb640 1 -- 172.21.15.163:0/877899004 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6f280604e0 con 0x7f6f3015f4c0 2024-05-08T06:36:54.829 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.830+0000 7f6f379da640 1 -- 172.21.15.163:0/877899004 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f300008d0 con 0x7f6f140762d0 2024-05-08T06:36:54.834 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.835+0000 7f6f25ffb640 1 -- 172.21.15.163:0/877899004 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f6f300008d0 con 0x7f6f140762d0 2024-05-08T06:36:54.837 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:54.838 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2024-05-08T06:36:54.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.839+0000 7f6f379da640 1 -- 172.21.15.163:0/877899004 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6f140762d0 msgr2=0x7f6f14078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.839+0000 7f6f379da640 1 --2- 172.21.15.163:0/877899004 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6f140762d0 0x7f6f14078790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c034000 tx=0x7f6f2c02e000 comp rx=0 tx=0).stop 2024-05-08T06:36:54.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.839+0000 7f6f379da640 1 -- 172.21.15.163:0/877899004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f3015f4c0 msgr2=0x7f6f30160320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.839+0000 7f6f379da640 1 --2- 172.21.15.163:0/877899004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f3015f4c0 0x7f6f30160320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800c7a0 tx=0x7f6f2800cc60 comp rx=0 tx=0).stop 2024-05-08T06:36:54.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.839+0000 7f6f379da640 1 -- 172.21.15.163:0/877899004 shutdown_connections 2024-05-08T06:36:54.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.839+0000 7f6f379da640 1 --2- 172.21.15.163:0/877899004 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6f140762d0 0x7f6f14078790 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.839+0000 7f6f379da640 1 --2- 172.21.15.163:0/877899004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f3015f4c0 0x7f6f30160320 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.839+0000 7f6f379da640 1 --2- 172.21.15.163:0/877899004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f300d3820 0x7f6f300c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.839+0000 7f6f379da640 1 --2- 172.21.15.163:0/877899004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f300baf20 0x7f6f301f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.839+0000 7f6f379da640 1 -- 172.21.15.163:0/877899004 >> 172.21.15.163:0/877899004 conn(0x7f6f30073e70 msgr2=0x7f6f300b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:54.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.839+0000 7f6f379da640 1 -- 172.21.15.163:0/877899004 shutdown_connections 2024-05-08T06:36:54.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.839+0000 7f6f379da640 1 -- 172.21.15.163:0/877899004 wait complete. 2024-05-08T06:36:54.849 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-05-08T06:36:54.100349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.452350"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109935+0000","last_change":"2024-05-08T06:36:41.664385+0000","last_active":"2024-05-08T06:36:43.109935+0000","last_peered":"2024-05-08T06:36:43.109935+0000","last_clean":"2024-05-08T06:36:43.109935+0000","last_became_active":"2024-05-08T06:36:41.664106+0000","last_became_peered":"2024-05-08T06:36:41.664106+0000","last_unstale":"2024-05-08T06:36:43.109935+0000","last_undegraded":"2024-05-08T06:36:43.109935+0000","last_fullsized":"2024-05-08T06:36:43.109935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:08.774451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649403+0000","last_change":"2024-05-08T06:36:41.647763+0000","last_active":"2024-05-08T06:36:42.649403+0000","last_peered":"2024-05-08T06:36:42.649403+0000","last_clean":"2024-05-08T06:36:42.649403+0000","last_became_active":"2024-05-08T06:36:41.647487+0000","last_became_peered":"2024-05-08T06:36:41.647487+0000","last_unstale":"2024-05-08T06:36:42.649403+0000","last_undegraded":"2024-05-08T06:36:42.649403+0000","last_fullsized":"2024-05-08T06:36:42.649403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108764+0000","last_change":"2024-05-08T06:36:41.649945+0000","last_active":"2024-05-08T06:36:43.108764+0000","last_peered":"2024-05-08T06:36:43.108764+0000","last_clean":"2024-05-08T06:36:43.108764+0000","last_became_active":"2024-05-08T06:36:41.649665+0000","last_became_peered":"2024-05-08T06:36:41.649665+0000","last_unstale":"2024-05-08T06:36:43.108764+0000","last_undegraded":"2024-05-08T06:36:43.108764+0000","last_fullsized":"2024-05-08T06:36:43.108764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109325+0000","last_change":"2024-05-08T06:36:41.656179+0000","last_active":"2024-05-08T06:36:43.109325+0000","last_peered":"2024-05-08T06:36:43.109325+0000","last_clean":"2024-05-08T06:36:43.109325+0000","last_became_active":"2024-05-08T06:36:41.655649+0000","last_became_peered":"2024-05-08T06:36:41.655649+0000","last_unstale":"2024-05-08T06:36:43.109325+0000","last_undegraded":"2024-05-08T06:36:43.109325+0000","last_fullsized":"2024-05-08T06:36:43.109325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:45:07.961465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109228+0000","last_change":"2024-05-08T06:36:41.659928+0000","last_active":"2024-05-08T06:36:43.109228+0000","last_peered":"2024-05-08T06:36:43.109228+0000","last_clean":"2024-05-08T06:36:43.109228+0000","last_became_active":"2024-05-08T06:36:41.659408+0000","last_became_peered":"2024-05-08T06:36:41.659408+0000","last_unstale":"2024-05-08T06:36:43.109228+0000","last_undegraded":"2024-05-08T06:36:43.109228+0000","last_fullsized":"2024-05-08T06:36:43.109228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:14.886723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109064+0000","last_change":"2024-05-08T06:36:41.653658+0000","last_active":"2024-05-08T06:36:43.109064+0000","last_peered":"2024-05-08T06:36:43.109064+0000","last_clean":"2024-05-08T06:36:43.109064+0000","last_became_active":"2024-05-08T06:36:41.653285+0000","last_became_peered":"2024-05-08T06:36:41.653285+0000","last_unstale":"2024-05-08T06:36:43.109064+0000","last_undegraded":"2024-05-08T06:36:43.109064+0000","last_fullsized":"2024-05-08T06:36:43.109064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108884+0000","last_change":"2024-05-08T06:36:41.661193+0000","last_active":"2024-05-08T06:36:43.108884+0000","last_peered":"2024-05-08T06:36:43.108884+0000","last_clean":"2024-05-08T06:36:43.108884+0000","last_became_active":"2024-05-08T06:36:41.660907+0000","last_became_peered":"2024-05-08T06:36:41.660907+0000","last_unstale":"2024-05-08T06:36:43.108884+0000","last_undegraded":"2024-05-08T06:36:43.108884+0000","last_fullsized":"2024-05-08T06:36:43.108884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108493+0000","last_change":"2024-05-08T06:36:41.655350+0000","last_active":"2024-05-08T06:36:43.108493+0000","last_peered":"2024-05-08T06:36:43.108493+0000","last_clean":"2024-05-08T06:36:43.108493+0000","last_became_active":"2024-05-08T06:36:41.655064+0000","last_became_peered":"2024-05-08T06:36:41.655064+0000","last_unstale":"2024-05-08T06:36:43.108493+0000","last_undegraded":"2024-05-08T06:36:43.108493+0000","last_fullsized":"2024-05-08T06:36:43.108493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:11:54.263628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109972+0000","last_change":"2024-05-08T06:36:41.663210+0000","last_active":"2024-05-08T06:36:43.109972+0000","last_peered":"2024-05-08T06:36:43.109972+0000","last_clean":"2024-05-08T06:36:43.109972+0000","last_became_active":"2024-05-08T06:36:41.662954+0000","last_became_peered":"2024-05-08T06:36:41.662954+0000","last_unstale":"2024-05-08T06:36:43.109972+0000","last_undegraded":"2024-05-08T06:36:43.109972+0000","last_fullsized":"2024-05-08T06:36:43.109972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:36:32.652292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654330+0000","last_change":"2024-05-08T06:36:41.662910+0000","last_active":"2024-05-08T06:36:42.654330+0000","last_peered":"2024-05-08T06:36:42.654330+0000","last_clean":"2024-05-08T06:36:42.654330+0000","last_became_active":"2024-05-08T06:36:41.662585+0000","last_became_peered":"2024-05-08T06:36:41.662585+0000","last_unstale":"2024-05-08T06:36:42.654330+0000","last_undegraded":"2024-05-08T06:36:42.654330+0000","last_fullsized":"2024-05-08T06:36:42.654330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653737+0000","last_change":"2024-05-08T06:36:41.663863+0000","last_active":"2024-05-08T06:36:42.653737+0000","last_peered":"2024-05-08T06:36:42.653737+0000","last_clean":"2024-05-08T06:36:42.653737+0000","last_became_active":"2024-05-08T06:36:41.663584+0000","last_became_peered":"2024-05-08T06:36:41.663584+0000","last_unstale":"2024-05-08T06:36:42.653737+0000","last_undegraded":"2024-05-08T06:36:42.653737+0000","last_fullsized":"2024-05-08T06:36:42.653737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654617+0000","last_change":"2024-05-08T06:36:41.661979+0000","last_active":"2024-05-08T06:36:42.654617+0000","last_peered":"2024-05-08T06:36:42.654617+0000","last_clean":"2024-05-08T06:36:42.654617+0000","last_became_active":"2024-05-08T06:36:41.660222+0000","last_became_peered":"2024-05-08T06:36:41.660222+0000","last_unstale":"2024-05-08T06:36:42.654617+0000","last_undegraded":"2024-05-08T06:36:42.654617+0000","last_fullsized":"2024-05-08T06:36:42.654617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:26.842934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108871+0000","last_change":"2024-05-08T06:36:41.654832+0000","last_active":"2024-05-08T06:36:43.108871+0000","last_peered":"2024-05-08T06:36:43.108871+0000","last_clean":"2024-05-08T06:36:43.108871+0000","last_became_active":"2024-05-08T06:36:41.654550+0000","last_became_peered":"2024-05-08T06:36:41.654550+0000","last_unstale":"2024-05-08T06:36:43.108871+0000","last_undegraded":"2024-05-08T06:36:43.108871+0000","last_fullsized":"2024-05-08T06:36:43.108871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108766+0000","last_change":"2024-05-08T06:36:41.642429+0000","last_active":"2024-05-08T06:36:43.108766+0000","last_peered":"2024-05-08T06:36:43.108766+0000","last_clean":"2024-05-08T06:36:43.108766+0000","last_became_active":"2024-05-08T06:36:41.641956+0000","last_became_peered":"2024-05-08T06:36:41.641956+0000","last_unstale":"2024-05-08T06:36:43.108766+0000","last_undegraded":"2024-05-08T06:36:43.108766+0000","last_fullsized":"2024-05-08T06:36:43.108766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649200+0000","last_change":"2024-05-08T06:36:41.656423+0000","last_active":"2024-05-08T06:36:42.649200+0000","last_peered":"2024-05-08T06:36:42.649200+0000","last_clean":"2024-05-08T06:36:42.649200+0000","last_became_active":"2024-05-08T06:36:41.656154+0000","last_became_peered":"2024-05-08T06:36:41.656154+0000","last_unstale":"2024-05-08T06:36:42.649200+0000","last_undegraded":"2024-05-08T06:36:42.649200+0000","last_fullsized":"2024-05-08T06:36:42.649200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109681+0000","last_change":"2024-05-08T06:36:41.653378+0000","last_active":"2024-05-08T06:36:43.109681+0000","last_peered":"2024-05-08T06:36:43.109681+0000","last_clean":"2024-05-08T06:36:43.109681+0000","last_became_active":"2024-05-08T06:36:41.653027+0000","last_became_peered":"2024-05-08T06:36:41.653027+0000","last_unstale":"2024-05-08T06:36:43.109681+0000","last_undegraded":"2024-05-08T06:36:43.109681+0000","last_fullsized":"2024-05-08T06:36:43.109681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:02.654155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109919+0000","last_change":"2024-05-08T06:36:41.659055+0000","last_active":"2024-05-08T06:36:43.109919+0000","last_peered":"2024-05-08T06:36:43.109919+0000","last_clean":"2024-05-08T06:36:43.109919+0000","last_became_active":"2024-05-08T06:36:41.658774+0000","last_became_peered":"2024-05-08T06:36:41.658774+0000","last_unstale":"2024-05-08T06:36:43.109919+0000","last_undegraded":"2024-05-08T06:36:43.109919+0000","last_fullsized":"2024-05-08T06:36:43.109919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:38:12.052561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110931+0000","last_change":"2024-05-08T06:36:41.649669+0000","last_active":"2024-05-08T06:36:43.110931+0000","last_peered":"2024-05-08T06:36:43.110931+0000","last_clean":"2024-05-08T06:36:43.110931+0000","last_became_active":"2024-05-08T06:36:41.649391+0000","last_became_peered":"2024-05-08T06:36:41.649391+0000","last_unstale":"2024-05-08T06:36:43.110931+0000","last_undegraded":"2024-05-08T06:36:43.110931+0000","last_fullsized":"2024-05-08T06:36:43.110931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:04.579668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109443+0000","last_change":"2024-05-08T06:36:41.660852+0000","last_active":"2024-05-08T06:36:43.109443+0000","last_peered":"2024-05-08T06:36:43.109443+0000","last_clean":"2024-05-08T06:36:43.109443+0000","last_became_active":"2024-05-08T06:36:41.660573+0000","last_became_peered":"2024-05-08T06:36:41.660573+0000","last_unstale":"2024-05-08T06:36:43.109443+0000","last_undegraded":"2024-05-08T06:36:43.109443+0000","last_fullsized":"2024-05-08T06:36:43.109443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:02.654155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653881+0000","last_change":"2024-05-08T06:36:41.663211+0000","last_active":"2024-05-08T06:36:42.653881+0000","last_peered":"2024-05-08T06:36:42.653881+0000","last_clean":"2024-05-08T06:36:42.653881+0000","last_became_active":"2024-05-08T06:36:41.662936+0000","last_became_peered":"2024-05-08T06:36:41.662936+0000","last_unstale":"2024-05-08T06:36:42.653881+0000","last_undegraded":"2024-05-08T06:36:42.653881+0000","last_fullsized":"2024-05-08T06:36:42.653881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108430+0000","last_change":"2024-05-08T06:36:41.661453+0000","last_active":"2024-05-08T06:36:43.108430+0000","last_peered":"2024-05-08T06:36:43.108430+0000","last_clean":"2024-05-08T06:36:43.108430+0000","last_became_active":"2024-05-08T06:36:41.661146+0000","last_became_peered":"2024-05-08T06:36:41.661146+0000","last_unstale":"2024-05-08T06:36:43.108430+0000","last_undegraded":"2024-05-08T06:36:43.108430+0000","last_fullsized":"2024-05-08T06:36:43.108430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:11:54.263628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109377+0000","last_change":"2024-05-08T06:36:41.658664+0000","last_active":"2024-05-08T06:36:43.109377+0000","last_peered":"2024-05-08T06:36:43.109377+0000","last_clean":"2024-05-08T06:36:43.109377+0000","last_became_active":"2024-05-08T06:36:41.658352+0000","last_became_peered":"2024-05-08T06:36:41.658352+0000","last_unstale":"2024-05-08T06:36:43.109377+0000","last_undegraded":"2024-05-08T06:36:43.109377+0000","last_fullsized":"2024-05-08T06:36:43.109377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108694+0000","last_change":"2024-05-08T06:36:41.658967+0000","last_active":"2024-05-08T06:36:43.108694+0000","last_peered":"2024-05-08T06:36:43.108694+0000","last_clean":"2024-05-08T06:36:43.108694+0000","last_became_active":"2024-05-08T06:36:41.658387+0000","last_became_peered":"2024-05-08T06:36:41.658387+0000","last_unstale":"2024-05-08T06:36:43.108694+0000","last_undegraded":"2024-05-08T06:36:43.108694+0000","last_fullsized":"2024-05-08T06:36:43.108694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108892+0000","last_change":"2024-05-08T06:36:41.657102+0000","last_active":"2024-05-08T06:36:43.108892+0000","last_peered":"2024-05-08T06:36:43.108892+0000","last_clean":"2024-05-08T06:36:43.108892+0000","last_became_active":"2024-05-08T06:36:41.656844+0000","last_became_peered":"2024-05-08T06:36:41.656844+0000","last_unstale":"2024-05-08T06:36:43.108892+0000","last_undegraded":"2024-05-08T06:36:43.108892+0000","last_fullsized":"2024-05-08T06:36:43.108892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109150+0000","last_change":"2024-05-08T06:36:41.659592+0000","last_active":"2024-05-08T06:36:43.109150+0000","last_peered":"2024-05-08T06:36:43.109150+0000","last_clean":"2024-05-08T06:36:43.109150+0000","last_became_active":"2024-05-08T06:36:41.659295+0000","last_became_peered":"2024-05-08T06:36:41.659295+0000","last_unstale":"2024-05-08T06:36:43.109150+0000","last_undegraded":"2024-05-08T06:36:43.109150+0000","last_fullsized":"2024-05-08T06:36:43.109150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654479+0000","last_change":"2024-05-08T06:36:41.649572+0000","last_active":"2024-05-08T06:36:42.654479+0000","last_peered":"2024-05-08T06:36:42.654479+0000","last_clean":"2024-05-08T06:36:42.654479+0000","last_became_active":"2024-05-08T06:36:41.649302+0000","last_became_peered":"2024-05-08T06:36:41.649302+0000","last_unstale":"2024-05-08T06:36:42.654479+0000","last_undegraded":"2024-05-08T06:36:42.654479+0000","last_fullsized":"2024-05-08T06:36:42.654479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649400+0000","last_change":"2024-05-08T06:36:41.657182+0000","last_active":"2024-05-08T06:36:42.649400+0000","last_peered":"2024-05-08T06:36:42.649400+0000","last_clean":"2024-05-08T06:36:42.649400+0000","last_became_active":"2024-05-08T06:36:41.656919+0000","last_became_peered":"2024-05-08T06:36:41.656919+0000","last_unstale":"2024-05-08T06:36:42.649400+0000","last_undegraded":"2024-05-08T06:36:42.649400+0000","last_fullsized":"2024-05-08T06:36:42.649400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109121+0000","last_change":"2024-05-08T06:36:41.659147+0000","last_active":"2024-05-08T06:36:43.109121+0000","last_peered":"2024-05-08T06:36:43.109121+0000","last_clean":"2024-05-08T06:36:43.109121+0000","last_became_active":"2024-05-08T06:36:41.658880+0000","last_became_peered":"2024-05-08T06:36:41.658880+0000","last_unstale":"2024-05-08T06:36:43.109121+0000","last_undegraded":"2024-05-08T06:36:43.109121+0000","last_fullsized":"2024-05-08T06:36:43.109121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108909+0000","last_change":"2024-05-08T06:36:41.658925+0000","last_active":"2024-05-08T06:36:43.108909+0000","last_peered":"2024-05-08T06:36:43.108909+0000","last_clean":"2024-05-08T06:36:43.108909+0000","last_became_active":"2024-05-08T06:36:41.656879+0000","last_became_peered":"2024-05-08T06:36:41.656879+0000","last_unstale":"2024-05-08T06:36:43.108909+0000","last_undegraded":"2024-05-08T06:36:43.108909+0000","last_fullsized":"2024-05-08T06:36:43.108909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109551+0000","last_change":"2024-05-08T06:36:41.655125+0000","last_active":"2024-05-08T06:36:43.109551+0000","last_peered":"2024-05-08T06:36:43.109551+0000","last_clean":"2024-05-08T06:36:43.109551+0000","last_became_active":"2024-05-08T06:36:41.654817+0000","last_became_peered":"2024-05-08T06:36:41.654817+0000","last_unstale":"2024-05-08T06:36:43.109551+0000","last_undegraded":"2024-05-08T06:36:43.109551+0000","last_fullsized":"2024-05-08T06:36:43.109551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:42.444444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110786+0000","last_change":"2024-05-08T06:36:41.655361+0000","last_active":"2024-05-08T06:36:43.110786+0000","last_peered":"2024-05-08T06:36:43.110786+0000","last_clean":"2024-05-08T06:36:43.110786+0000","last_became_active":"2024-05-08T06:36:41.655085+0000","last_became_peered":"2024-05-08T06:36:41.655085+0000","last_unstale":"2024-05-08T06:36:43.110786+0000","last_undegraded":"2024-05-08T06:36:43.110786+0000","last_fullsized":"2024-05-08T06:36:43.110786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:26.842934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653569+0000","last_change":"2024-05-08T06:36:41.643791+0000","last_active":"2024-05-08T06:36:42.653569+0000","last_peered":"2024-05-08T06:36:42.653569+0000","last_clean":"2024-05-08T06:36:42.653569+0000","last_became_active":"2024-05-08T06:36:41.643506+0000","last_became_peered":"2024-05-08T06:36:41.643506+0000","last_unstale":"2024-05-08T06:36:42.653569+0000","last_undegraded":"2024-05-08T06:36:42.653569+0000","last_fullsized":"2024-05-08T06:36:42.653569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654375+0000","last_change":"2024-05-08T06:36:41.654106+0000","last_active":"2024-05-08T06:36:42.654375+0000","last_peered":"2024-05-08T06:36:42.654375+0000","last_clean":"2024-05-08T06:36:42.654375+0000","last_became_active":"2024-05-08T06:36:41.653822+0000","last_became_peered":"2024-05-08T06:36:41.653822+0000","last_unstale":"2024-05-08T06:36:42.654375+0000","last_undegraded":"2024-05-08T06:36:42.654375+0000","last_fullsized":"2024-05-08T06:36:42.654375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:51.358786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109209+0000","last_change":"2024-05-08T06:36:40.641349+0000","last_active":"2024-05-08T06:36:43.109209+0000","last_peered":"2024-05-08T06:36:43.109209+0000","last_clean":"2024-05-08T06:36:43.109209+0000","last_became_active":"2024-05-08T06:36:40.641086+0000","last_became_peered":"2024-05-08T06:36:40.641086+0000","last_unstale":"2024-05-08T06:36:43.109209+0000","last_undegraded":"2024-05-08T06:36:43.109209+0000","last_fullsized":"2024-05-08T06:36:43.109209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:39:09.616149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108579+0000","last_change":"2024-05-08T06:36:41.656120+0000","last_active":"2024-05-08T06:36:43.108579+0000","last_peered":"2024-05-08T06:36:43.108579+0000","last_clean":"2024-05-08T06:36:43.108579+0000","last_became_active":"2024-05-08T06:36:41.655843+0000","last_became_peered":"2024-05-08T06:36:41.655843+0000","last_unstale":"2024-05-08T06:36:43.108579+0000","last_undegraded":"2024-05-08T06:36:43.108579+0000","last_fullsized":"2024-05-08T06:36:43.108579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654066+0000","last_change":"2024-05-08T06:36:40.639740+0000","last_active":"2024-05-08T06:36:42.654066+0000","last_peered":"2024-05-08T06:36:42.654066+0000","last_clean":"2024-05-08T06:36:42.654066+0000","last_became_active":"2024-05-08T06:36:40.639370+0000","last_became_peered":"2024-05-08T06:36:40.639370+0000","last_unstale":"2024-05-08T06:36:42.654066+0000","last_undegraded":"2024-05-08T06:36:42.654066+0000","last_fullsized":"2024-05-08T06:36:42.654066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108684+0000","last_change":"2024-05-08T06:36:40.641983+0000","last_active":"2024-05-08T06:36:43.108684+0000","last_peered":"2024-05-08T06:36:43.108684+0000","last_clean":"2024-05-08T06:36:43.108684+0000","last_became_active":"2024-05-08T06:36:40.641666+0000","last_became_peered":"2024-05-08T06:36:40.641666+0000","last_unstale":"2024-05-08T06:36:43.108684+0000","last_undegraded":"2024-05-08T06:36:43.108684+0000","last_fullsized":"2024-05-08T06:36:43.108684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:47.997246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108516+0000","last_change":"2024-05-08T06:36:41.653257+0000","last_active":"2024-05-08T06:36:43.108516+0000","last_peered":"2024-05-08T06:36:43.108516+0000","last_clean":"2024-05-08T06:36:43.108516+0000","last_became_active":"2024-05-08T06:36:41.652900+0000","last_became_peered":"2024-05-08T06:36:41.652900+0000","last_unstale":"2024-05-08T06:36:43.108516+0000","last_undegraded":"2024-05-08T06:36:43.108516+0000","last_fullsized":"2024-05-08T06:36:43.108516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109272+0000","last_change":"2024-05-08T06:36:40.646856+0000","last_active":"2024-05-08T06:36:43.109272+0000","last_peered":"2024-05-08T06:36:43.109272+0000","last_clean":"2024-05-08T06:36:43.109272+0000","last_became_active":"2024-05-08T06:36:40.644640+0000","last_became_peered":"2024-05-08T06:36:40.644640+0000","last_unstale":"2024-05-08T06:36:43.109272+0000","last_undegraded":"2024-05-08T06:36:43.109272+0000","last_fullsized":"2024-05-08T06:36:43.109272+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649183+0000","last_change":"2024-05-08T06:36:41.656744+0000","last_active":"2024-05-08T06:36:42.649183+0000","last_peered":"2024-05-08T06:36:42.649183+0000","last_clean":"2024-05-08T06:36:42.649183+0000","last_became_active":"2024-05-08T06:36:41.656483+0000","last_became_peered":"2024-05-08T06:36:41.656483+0000","last_unstale":"2024-05-08T06:36:42.649183+0000","last_undegraded":"2024-05-08T06:36:42.649183+0000","last_fullsized":"2024-05-08T06:36:42.649183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653768+0000","last_change":"2024-05-08T06:36:40.648551+0000","last_active":"2024-05-08T06:36:42.653768+0000","last_peered":"2024-05-08T06:36:42.653768+0000","last_clean":"2024-05-08T06:36:42.653768+0000","last_became_active":"2024-05-08T06:36:40.648265+0000","last_became_peered":"2024-05-08T06:36:40.648265+0000","last_unstale":"2024-05-08T06:36:42.653768+0000","last_undegraded":"2024-05-08T06:36:42.653768+0000","last_fullsized":"2024-05-08T06:36:42.653768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109024+0000","last_change":"2024-05-08T06:36:41.658637+0000","last_active":"2024-05-08T06:36:43.109024+0000","last_peered":"2024-05-08T06:36:43.109024+0000","last_clean":"2024-05-08T06:36:43.109024+0000","last_became_active":"2024-05-08T06:36:41.658333+0000","last_became_peered":"2024-05-08T06:36:41.658333+0000","last_unstale":"2024-05-08T06:36:43.109024+0000","last_undegraded":"2024-05-08T06:36:43.109024+0000","last_fullsized":"2024-05-08T06:36:43.109024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:31.061278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109045+0000","last_change":"2024-05-08T06:36:40.647645+0000","last_active":"2024-05-08T06:36:43.109045+0000","last_peered":"2024-05-08T06:36:43.109045+0000","last_clean":"2024-05-08T06:36:43.109045+0000","last_became_active":"2024-05-08T06:36:40.647368+0000","last_became_peered":"2024-05-08T06:36:40.647368+0000","last_unstale":"2024-05-08T06:36:43.109045+0000","last_undegraded":"2024-05-08T06:36:43.109045+0000","last_fullsized":"2024-05-08T06:36:43.109045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109122+0000","last_change":"2024-05-08T06:36:41.662012+0000","last_active":"2024-05-08T06:36:43.109122+0000","last_peered":"2024-05-08T06:36:43.109122+0000","last_clean":"2024-05-08T06:36:43.109122+0000","last_became_active":"2024-05-08T06:36:41.661780+0000","last_became_peered":"2024-05-08T06:36:41.661780+0000","last_unstale":"2024-05-08T06:36:43.109122+0000","last_undegraded":"2024-05-08T06:36:43.109122+0000","last_fullsized":"2024-05-08T06:36:43.109122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:07.298536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649372+0000","last_change":"2024-05-08T06:36:40.649405+0000","last_active":"2024-05-08T06:36:42.649372+0000","last_peered":"2024-05-08T06:36:42.649372+0000","last_clean":"2024-05-08T06:36:42.649372+0000","last_became_active":"2024-05-08T06:36:40.649172+0000","last_became_peered":"2024-05-08T06:36:40.649172+0000","last_unstale":"2024-05-08T06:36:42.649372+0000","last_undegraded":"2024-05-08T06:36:42.649372+0000","last_fullsized":"2024-05-08T06:36:42.649372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654679+0000","last_change":"2024-05-08T06:36:41.654141+0000","last_active":"2024-05-08T06:36:42.654679+0000","last_peered":"2024-05-08T06:36:42.654679+0000","last_clean":"2024-05-08T06:36:42.654679+0000","last_became_active":"2024-05-08T06:36:41.653723+0000","last_became_peered":"2024-05-08T06:36:41.653723+0000","last_unstale":"2024-05-08T06:36:42.654679+0000","last_undegraded":"2024-05-08T06:36:42.654679+0000","last_fullsized":"2024-05-08T06:36:42.654679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:04.579668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654000+0000","last_change":"2024-05-08T06:36:40.641783+0000","last_active":"2024-05-08T06:36:42.654000+0000","last_peered":"2024-05-08T06:36:42.654000+0000","last_clean":"2024-05-08T06:36:42.654000+0000","last_became_active":"2024-05-08T06:36:40.641270+0000","last_became_peered":"2024-05-08T06:36:40.641270+0000","last_unstale":"2024-05-08T06:36:42.654000+0000","last_undegraded":"2024-05-08T06:36:42.654000+0000","last_fullsized":"2024-05-08T06:36:42.654000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108979+0000","last_change":"2024-05-08T06:36:41.657579+0000","last_active":"2024-05-08T06:36:43.108979+0000","last_peered":"2024-05-08T06:36:43.108979+0000","last_clean":"2024-05-08T06:36:43.108979+0000","last_became_active":"2024-05-08T06:36:41.657299+0000","last_became_peered":"2024-05-08T06:36:41.657299+0000","last_unstale":"2024-05-08T06:36:43.108979+0000","last_undegraded":"2024-05-08T06:36:43.108979+0000","last_fullsized":"2024-05-08T06:36:43.108979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:02.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108685+0000","last_change":"2024-05-08T06:36:40.641651+0000","last_active":"2024-05-08T06:36:43.108685+0000","last_peered":"2024-05-08T06:36:43.108685+0000","last_clean":"2024-05-08T06:36:43.108685+0000","last_became_active":"2024-05-08T06:36:40.641168+0000","last_became_peered":"2024-05-08T06:36:40.641168+0000","last_unstale":"2024-05-08T06:36:43.108685+0000","last_undegraded":"2024-05-08T06:36:43.108685+0000","last_fullsized":"2024-05-08T06:36:43.108685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:47.997246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108903+0000","last_change":"2024-05-08T06:36:41.661356+0000","last_active":"2024-05-08T06:36:43.108903+0000","last_peered":"2024-05-08T06:36:43.108903+0000","last_clean":"2024-05-08T06:36:43.108903+0000","last_became_active":"2024-05-08T06:36:41.661085+0000","last_became_peered":"2024-05-08T06:36:41.661085+0000","last_unstale":"2024-05-08T06:36:43.108903+0000","last_undegraded":"2024-05-08T06:36:43.108903+0000","last_fullsized":"2024-05-08T06:36:43.108903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109770+0000","last_change":"2024-05-08T06:36:40.646694+0000","last_active":"2024-05-08T06:36:43.109770+0000","last_peered":"2024-05-08T06:36:43.109770+0000","last_clean":"2024-05-08T06:36:43.109770+0000","last_became_active":"2024-05-08T06:36:40.645833+0000","last_became_peered":"2024-05-08T06:36:40.645833+0000","last_unstale":"2024-05-08T06:36:43.109770+0000","last_undegraded":"2024-05-08T06:36:43.109770+0000","last_fullsized":"2024-05-08T06:36:43.109770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109418+0000","last_change":"2024-05-08T06:36:41.668787+0000","last_active":"2024-05-08T06:36:43.109418+0000","last_peered":"2024-05-08T06:36:43.109418+0000","last_clean":"2024-05-08T06:36:43.109418+0000","last_became_active":"2024-05-08T06:36:41.668489+0000","last_became_peered":"2024-05-08T06:36:41.668489+0000","last_unstale":"2024-05-08T06:36:43.109418+0000","last_undegraded":"2024-05-08T06:36:43.109418+0000","last_fullsized":"2024-05-08T06:36:43.109418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.395237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653600+0000","last_change":"2024-05-08T06:36:40.640082+0000","last_active":"2024-05-08T06:36:42.653600+0000","last_peered":"2024-05-08T06:36:42.653600+0000","last_clean":"2024-05-08T06:36:42.653600+0000","last_became_active":"2024-05-08T06:36:40.638558+0000","last_became_peered":"2024-05-08T06:36:40.638558+0000","last_unstale":"2024-05-08T06:36:42.653600+0000","last_undegraded":"2024-05-08T06:36:42.653600+0000","last_fullsized":"2024-05-08T06:36:42.653600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653523+0000","last_change":"2024-05-08T06:36:41.643855+0000","last_active":"2024-05-08T06:36:42.653523+0000","last_peered":"2024-05-08T06:36:42.653523+0000","last_clean":"2024-05-08T06:36:42.653523+0000","last_became_active":"2024-05-08T06:36:41.643117+0000","last_became_peered":"2024-05-08T06:36:41.643117+0000","last_unstale":"2024-05-08T06:36:42.653523+0000","last_undegraded":"2024-05-08T06:36:42.653523+0000","last_fullsized":"2024-05-08T06:36:42.653523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110713+0000","last_change":"2024-05-08T06:36:40.642303+0000","last_active":"2024-05-08T06:36:43.110713+0000","last_peered":"2024-05-08T06:36:43.110713+0000","last_clean":"2024-05-08T06:36:43.110713+0000","last_became_active":"2024-05-08T06:36:40.642042+0000","last_became_peered":"2024-05-08T06:36:40.642042+0000","last_unstale":"2024-05-08T06:36:43.110713+0000","last_undegraded":"2024-05-08T06:36:43.110713+0000","last_fullsized":"2024-05-08T06:36:43.110713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:28.086048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109058+0000","last_change":"2024-05-08T06:36:41.660559+0000","last_active":"2024-05-08T06:36:43.109058+0000","last_peered":"2024-05-08T06:36:43.109058+0000","last_clean":"2024-05-08T06:36:43.109058+0000","last_became_active":"2024-05-08T06:36:41.660282+0000","last_became_peered":"2024-05-08T06:36:41.660282+0000","last_unstale":"2024-05-08T06:36:43.109058+0000","last_undegraded":"2024-05-08T06:36:43.109058+0000","last_fullsized":"2024-05-08T06:36:43.109058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:49.740862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654374+0000","last_change":"2024-05-08T06:36:40.646655+0000","last_active":"2024-05-08T06:36:42.654374+0000","last_peered":"2024-05-08T06:36:42.654374+0000","last_clean":"2024-05-08T06:36:42.654374+0000","last_became_active":"2024-05-08T06:36:40.646346+0000","last_became_peered":"2024-05-08T06:36:40.646346+0000","last_unstale":"2024-05-08T06:36:42.654374+0000","last_undegraded":"2024-05-08T06:36:42.654374+0000","last_fullsized":"2024-05-08T06:36:42.654374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109890+0000","last_change":"2024-05-08T06:36:41.659817+0000","last_active":"2024-05-08T06:36:43.109890+0000","last_peered":"2024-05-08T06:36:43.109890+0000","last_clean":"2024-05-08T06:36:43.109890+0000","last_became_active":"2024-05-08T06:36:41.659524+0000","last_became_peered":"2024-05-08T06:36:41.659524+0000","last_unstale":"2024-05-08T06:36:43.109890+0000","last_undegraded":"2024-05-08T06:36:43.109890+0000","last_fullsized":"2024-05-08T06:36:43.109890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:38:12.052561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110881+0000","last_change":"2024-05-08T06:36:40.641491+0000","last_active":"2024-05-08T06:36:43.110881+0000","last_peered":"2024-05-08T06:36:43.110881+0000","last_clean":"2024-05-08T06:36:43.110881+0000","last_became_active":"2024-05-08T06:36:40.641207+0000","last_became_peered":"2024-05-08T06:36:40.641207+0000","last_unstale":"2024-05-08T06:36:43.110881+0000","last_undegraded":"2024-05-08T06:36:43.110881+0000","last_fullsized":"2024-05-08T06:36:43.110881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110049+0000","last_change":"2024-05-08T06:36:41.667992+0000","last_active":"2024-05-08T06:36:43.110049+0000","last_peered":"2024-05-08T06:36:43.110049+0000","last_clean":"2024-05-08T06:36:43.110049+0000","last_became_active":"2024-05-08T06:36:41.667733+0000","last_became_peered":"2024-05-08T06:36:41.667733+0000","last_unstale":"2024-05-08T06:36:43.110049+0000","last_undegraded":"2024-05-08T06:36:43.110049+0000","last_fullsized":"2024-05-08T06:36:43.110049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:41:00.520597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649448+0000","last_change":"2024-05-08T06:36:40.646069+0000","last_active":"2024-05-08T06:36:42.649448+0000","last_peered":"2024-05-08T06:36:42.649448+0000","last_clean":"2024-05-08T06:36:42.649448+0000","last_became_active":"2024-05-08T06:36:40.645769+0000","last_became_peered":"2024-05-08T06:36:40.645769+0000","last_unstale":"2024-05-08T06:36:42.649448+0000","last_undegraded":"2024-05-08T06:36:42.649448+0000","last_fullsized":"2024-05-08T06:36:42.649448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:25.837342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109628+0000","last_change":"2024-05-08T06:36:41.660346+0000","last_active":"2024-05-08T06:36:43.109628+0000","last_peered":"2024-05-08T06:36:43.109628+0000","last_clean":"2024-05-08T06:36:43.109628+0000","last_became_active":"2024-05-08T06:36:41.660028+0000","last_became_peered":"2024-05-08T06:36:41.660028+0000","last_unstale":"2024-05-08T06:36:43.109628+0000","last_undegraded":"2024-05-08T06:36:43.109628+0000","last_fullsized":"2024-05-08T06:36:43.109628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:14.886723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110847+0000","last_change":"2024-05-08T06:36:40.647112+0000","last_active":"2024-05-08T06:36:43.110847+0000","last_peered":"2024-05-08T06:36:43.110847+0000","last_clean":"2024-05-08T06:36:43.110847+0000","last_became_active":"2024-05-08T06:36:40.646804+0000","last_became_peered":"2024-05-08T06:36:40.646804+0000","last_unstale":"2024-05-08T06:36:43.110847+0000","last_undegraded":"2024-05-08T06:36:43.110847+0000","last_fullsized":"2024-05-08T06:36:43.110847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109148+0000","last_change":"2024-05-08T06:36:41.663875+0000","last_active":"2024-05-08T06:36:43.109148+0000","last_peered":"2024-05-08T06:36:43.109148+0000","last_clean":"2024-05-08T06:36:43.109148+0000","last_became_active":"2024-05-08T06:36:41.663595+0000","last_became_peered":"2024-05-08T06:36:41.663595+0000","last_unstale":"2024-05-08T06:36:43.109148+0000","last_undegraded":"2024-05-08T06:36:43.109148+0000","last_fullsized":"2024-05-08T06:36:43.109148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:42.444444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108725+0000","last_change":"2024-05-08T06:36:31.710287+0000","last_active":"2024-05-08T06:36:43.108725+0000","last_peered":"2024-05-08T06:36:43.108725+0000","last_clean":"2024-05-08T06:36:43.108725+0000","last_became_active":"2024-05-08T06:36:31.709270+0000","last_became_peered":"2024-05-08T06:36:31.709270+0000","last_unstale":"2024-05-08T06:36:43.108725+0000","last_undegraded":"2024-05-08T06:36:43.108725+0000","last_fullsized":"2024-05-08T06:36:43.108725+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_clean_scrub_stamp":"2024-05-08T06:36:30.122918+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:42.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108783+0000","last_change":"2024-05-08T06:36:40.645536+0000","last_active":"2024-05-08T06:36:43.108783+0000","last_peered":"2024-05-08T06:36:43.108783+0000","last_clean":"2024-05-08T06:36:43.108783+0000","last_became_active":"2024-05-08T06:36:40.644256+0000","last_became_peered":"2024-05-08T06:36:40.644256+0000","last_unstale":"2024-05-08T06:36:43.108783+0000","last_undegraded":"2024-05-08T06:36:43.108783+0000","last_fullsized":"2024-05-08T06:36:43.108783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:48.949613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109026+0000","last_change":"2024-05-08T06:36:41.655301+0000","last_active":"2024-05-08T06:36:43.109026+0000","last_peered":"2024-05-08T06:36:43.109026+0000","last_clean":"2024-05-08T06:36:43.109026+0000","last_became_active":"2024-05-08T06:36:41.655090+0000","last_became_peered":"2024-05-08T06:36:41.655090+0000","last_unstale":"2024-05-08T06:36:43.109026+0000","last_undegraded":"2024-05-08T06:36:43.109026+0000","last_fullsized":"2024-05-08T06:36:43.109026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108653+0000","last_change":"2024-05-08T06:36:40.647913+0000","last_active":"2024-05-08T06:36:43.108653+0000","last_peered":"2024-05-08T06:36:43.108653+0000","last_clean":"2024-05-08T06:36:43.108653+0000","last_became_active":"2024-05-08T06:36:40.647638+0000","last_became_peered":"2024-05-08T06:36:40.647638+0000","last_unstale":"2024-05-08T06:36:43.108653+0000","last_undegraded":"2024-05-08T06:36:43.108653+0000","last_fullsized":"2024-05-08T06:36:43.108653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:29:44.646218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108285+0000","last_change":"2024-05-08T06:36:41.658563+0000","last_active":"2024-05-08T06:36:43.108285+0000","last_peered":"2024-05-08T06:36:43.108285+0000","last_clean":"2024-05-08T06:36:43.108285+0000","last_became_active":"2024-05-08T06:36:41.657157+0000","last_became_peered":"2024-05-08T06:36:41.657157+0000","last_unstale":"2024-05-08T06:36:43.108285+0000","last_undegraded":"2024-05-08T06:36:43.108285+0000","last_fullsized":"2024-05-08T06:36:43.108285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:30:14.798179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.110812+0000","last_change":"2024-05-08T06:36:40.642187+0000","last_active":"2024-05-08T06:36:43.110812+0000","last_peered":"2024-05-08T06:36:43.110812+0000","last_clean":"2024-05-08T06:36:43.110812+0000","last_became_active":"2024-05-08T06:36:40.641906+0000","last_became_peered":"2024-05-08T06:36:40.641906+0000","last_unstale":"2024-05-08T06:36:43.110812+0000","last_undegraded":"2024-05-08T06:36:43.110812+0000","last_fullsized":"2024-05-08T06:36:43.110812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649176+0000","last_change":"2024-05-08T06:36:41.642328+0000","last_active":"2024-05-08T06:36:42.649176+0000","last_peered":"2024-05-08T06:36:42.649176+0000","last_clean":"2024-05-08T06:36:42.649176+0000","last_became_active":"2024-05-08T06:36:41.642053+0000","last_became_peered":"2024-05-08T06:36:41.642053+0000","last_unstale":"2024-05-08T06:36:42.649176+0000","last_undegraded":"2024-05-08T06:36:42.649176+0000","last_fullsized":"2024-05-08T06:36:42.649176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:32.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109887+0000","last_change":"2024-05-08T06:36:40.646469+0000","last_active":"2024-05-08T06:36:43.109887+0000","last_peered":"2024-05-08T06:36:43.109887+0000","last_clean":"2024-05-08T06:36:43.109887+0000","last_became_active":"2024-05-08T06:36:40.646207+0000","last_became_peered":"2024-05-08T06:36:40.646207+0000","last_unstale":"2024-05-08T06:36:43.109887+0000","last_undegraded":"2024-05-08T06:36:43.109887+0000","last_fullsized":"2024-05-08T06:36:43.109887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654706+0000","last_change":"2024-05-08T06:36:41.654729+0000","last_active":"2024-05-08T06:36:42.654706+0000","last_peered":"2024-05-08T06:36:42.654706+0000","last_clean":"2024-05-08T06:36:42.654706+0000","last_became_active":"2024-05-08T06:36:41.654466+0000","last_became_peered":"2024-05-08T06:36:41.654466+0000","last_unstale":"2024-05-08T06:36:42.654706+0000","last_undegraded":"2024-05-08T06:36:42.654706+0000","last_fullsized":"2024-05-08T06:36:42.654706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:44.798191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649223+0000","last_change":"2024-05-08T06:36:40.647304+0000","last_active":"2024-05-08T06:36:42.649223+0000","last_peered":"2024-05-08T06:36:42.649223+0000","last_clean":"2024-05-08T06:36:42.649223+0000","last_became_active":"2024-05-08T06:36:40.647010+0000","last_became_peered":"2024-05-08T06:36:40.647010+0000","last_unstale":"2024-05-08T06:36:42.649223+0000","last_undegraded":"2024-05-08T06:36:42.649223+0000","last_fullsized":"2024-05-08T06:36:42.649223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:50.353194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109655+0000","last_change":"2024-05-08T06:36:41.656064+0000","last_active":"2024-05-08T06:36:43.109655+0000","last_peered":"2024-05-08T06:36:43.109655+0000","last_clean":"2024-05-08T06:36:43.109655+0000","last_became_active":"2024-05-08T06:36:41.655714+0000","last_became_peered":"2024-05-08T06:36:41.655714+0000","last_unstale":"2024-05-08T06:36:43.109655+0000","last_undegraded":"2024-05-08T06:36:43.109655+0000","last_fullsized":"2024-05-08T06:36:43.109655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.395237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109737+0000","last_change":"2024-05-08T06:36:40.648662+0000","last_active":"2024-05-08T06:36:43.109737+0000","last_peered":"2024-05-08T06:36:43.109737+0000","last_clean":"2024-05-08T06:36:43.109737+0000","last_became_active":"2024-05-08T06:36:40.648367+0000","last_became_peered":"2024-05-08T06:36:40.648367+0000","last_unstale":"2024-05-08T06:36:43.109737+0000","last_undegraded":"2024-05-08T06:36:43.109737+0000","last_fullsized":"2024-05-08T06:36:43.109737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:46:25.837342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649254+0000","last_change":"2024-05-08T06:36:41.642330+0000","last_active":"2024-05-08T06:36:42.649254+0000","last_peered":"2024-05-08T06:36:42.649254+0000","last_clean":"2024-05-08T06:36:42.649254+0000","last_became_active":"2024-05-08T06:36:41.642059+0000","last_became_peered":"2024-05-08T06:36:41.642059+0000","last_unstale":"2024-05-08T06:36:42.649254+0000","last_undegraded":"2024-05-08T06:36:42.649254+0000","last_fullsized":"2024-05-08T06:36:42.649254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649350+0000","last_change":"2024-05-08T06:36:40.646675+0000","last_active":"2024-05-08T06:36:42.649350+0000","last_peered":"2024-05-08T06:36:42.649350+0000","last_clean":"2024-05-08T06:36:42.649350+0000","last_became_active":"2024-05-08T06:36:40.646346+0000","last_became_peered":"2024-05-08T06:36:40.646346+0000","last_unstale":"2024-05-08T06:36:42.649350+0000","last_undegraded":"2024-05-08T06:36:42.649350+0000","last_fullsized":"2024-05-08T06:36:42.649350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:03.574076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109189+0000","last_change":"2024-05-08T06:36:41.658633+0000","last_active":"2024-05-08T06:36:43.109189+0000","last_peered":"2024-05-08T06:36:43.109189+0000","last_clean":"2024-05-08T06:36:43.109189+0000","last_became_active":"2024-05-08T06:36:41.658294+0000","last_became_peered":"2024-05-08T06:36:41.658294+0000","last_unstale":"2024-05-08T06:36:43.109189+0000","last_undegraded":"2024-05-08T06:36:43.109189+0000","last_fullsized":"2024-05-08T06:36:43.109189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:14.438762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653882+0000","last_change":"2024-05-08T06:36:40.639547+0000","last_active":"2024-05-08T06:36:42.653882+0000","last_peered":"2024-05-08T06:36:42.653882+0000","last_clean":"2024-05-08T06:36:42.653882+0000","last_became_active":"2024-05-08T06:36:40.639240+0000","last_became_peered":"2024-05-08T06:36:40.639240+0000","last_unstale":"2024-05-08T06:36:42.653882+0000","last_undegraded":"2024-05-08T06:36:42.653882+0000","last_fullsized":"2024-05-08T06:36:42.653882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108530+0000","last_change":"2024-05-08T06:36:41.648709+0000","last_active":"2024-05-08T06:36:43.108530+0000","last_peered":"2024-05-08T06:36:43.108530+0000","last_clean":"2024-05-08T06:36:43.108530+0000","last_became_active":"2024-05-08T06:36:41.648406+0000","last_became_peered":"2024-05-08T06:36:41.648406+0000","last_unstale":"2024-05-08T06:36:43.108530+0000","last_undegraded":"2024-05-08T06:36:43.108530+0000","last_fullsized":"2024-05-08T06:36:43.108530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:13.502665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108653+0000","last_change":"2024-05-08T06:36:40.648721+0000","last_active":"2024-05-08T06:36:43.108653+0000","last_peered":"2024-05-08T06:36:43.108653+0000","last_clean":"2024-05-08T06:36:43.108653+0000","last_became_active":"2024-05-08T06:36:40.648429+0000","last_became_peered":"2024-05-08T06:36:40.648429+0000","last_unstale":"2024-05-08T06:36:43.108653+0000","last_undegraded":"2024-05-08T06:36:43.108653+0000","last_fullsized":"2024-05-08T06:36:43.108653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:43.792599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.654290+0000","last_change":"2024-05-08T06:36:41.655289+0000","last_active":"2024-05-08T06:36:42.654290+0000","last_peered":"2024-05-08T06:36:42.654290+0000","last_clean":"2024-05-08T06:36:42.654290+0000","last_became_active":"2024-05-08T06:36:41.655022+0000","last_became_peered":"2024-05-08T06:36:41.655022+0000","last_unstale":"2024-05-08T06:36:42.654290+0000","last_undegraded":"2024-05-08T06:36:42.654290+0000","last_fullsized":"2024-05-08T06:36:42.654290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:57.405934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649256+0000","last_change":"2024-05-08T06:36:40.645335+0000","last_active":"2024-05-08T06:36:42.649256+0000","last_peered":"2024-05-08T06:36:42.649256+0000","last_clean":"2024-05-08T06:36:42.649256+0000","last_became_active":"2024-05-08T06:36:40.645019+0000","last_became_peered":"2024-05-08T06:36:40.645019+0000","last_unstale":"2024-05-08T06:36:42.649256+0000","last_undegraded":"2024-05-08T06:36:42.649256+0000","last_fullsized":"2024-05-08T06:36:42.649256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:14.581267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108851+0000","last_change":"2024-05-08T06:36:41.653910+0000","last_active":"2024-05-08T06:36:43.108851+0000","last_peered":"2024-05-08T06:36:43.108851+0000","last_clean":"2024-05-08T06:36:43.108851+0000","last_became_active":"2024-05-08T06:36:41.653567+0000","last_became_peered":"2024-05-08T06:36:41.653567+0000","last_unstale":"2024-05-08T06:36:43.108851+0000","last_undegraded":"2024-05-08T06:36:43.108851+0000","last_fullsized":"2024-05-08T06:36:43.108851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:58.769151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.649426+0000","last_change":"2024-05-08T06:36:40.642880+0000","last_active":"2024-05-08T06:36:42.649426+0000","last_peered":"2024-05-08T06:36:42.649426+0000","last_clean":"2024-05-08T06:36:42.649426+0000","last_became_active":"2024-05-08T06:36:40.642605+0000","last_became_peered":"2024-05-08T06:36:40.642605+0000","last_unstale":"2024-05-08T06:36:42.649426+0000","last_undegraded":"2024-05-08T06:36:42.649426+0000","last_fullsized":"2024-05-08T06:36:42.649426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:28:42.521220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109118+0000","last_change":"2024-05-08T06:36:41.654124+0000","last_active":"2024-05-08T06:36:43.109118+0000","last_peered":"2024-05-08T06:36:43.109118+0000","last_clean":"2024-05-08T06:36:43.109118+0000","last_became_active":"2024-05-08T06:36:41.653837+0000","last_became_peered":"2024-05-08T06:36:41.653837+0000","last_unstale":"2024-05-08T06:36:43.109118+0000","last_undegraded":"2024-05-08T06:36:43.109118+0000","last_fullsized":"2024-05-08T06:36:43.109118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:54.357999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109611+0000","last_change":"2024-05-08T06:36:40.644938+0000","last_active":"2024-05-08T06:36:43.109611+0000","last_peered":"2024-05-08T06:36:43.109611+0000","last_clean":"2024-05-08T06:36:43.109611+0000","last_became_active":"2024-05-08T06:36:40.644549+0000","last_became_peered":"2024-05-08T06:36:40.644549+0000","last_unstale":"2024-05-08T06:36:43.109611+0000","last_undegraded":"2024-05-08T06:36:43.109611+0000","last_fullsized":"2024-05-08T06:36:43.109611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:50.353194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:42.653954+0000","last_change":"2024-05-08T06:36:41.659401+0000","last_active":"2024-05-08T06:36:42.653954+0000","last_peered":"2024-05-08T06:36:42.653954+0000","last_clean":"2024-05-08T06:36:42.653954+0000","last_became_active":"2024-05-08T06:36:41.659063+0000","last_became_peered":"2024-05-08T06:36:41.659063+0000","last_unstale":"2024-05-08T06:36:42.653954+0000","last_undegraded":"2024-05-08T06:36:42.653954+0000","last_fullsized":"2024-05-08T06:36:42.653954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:28.015358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109449+0000","last_change":"2024-05-08T06:36:41.655858+0000","last_active":"2024-05-08T06:36:43.109449+0000","last_peered":"2024-05-08T06:36:43.109449+0000","last_clean":"2024-05-08T06:36:43.109449+0000","last_became_active":"2024-05-08T06:36:41.655569+0000","last_became_peered":"2024-05-08T06:36:41.655569+0000","last_unstale":"2024-05-08T06:36:43.109449+0000","last_undegraded":"2024-05-08T06:36:43.109449+0000","last_fullsized":"2024-05-08T06:36:43.109449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:02.239819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109476+0000","last_change":"2024-05-08T06:36:40.646643+0000","last_active":"2024-05-08T06:36:43.109476+0000","last_peered":"2024-05-08T06:36:43.109476+0000","last_clean":"2024-05-08T06:36:43.109476+0000","last_became_active":"2024-05-08T06:36:40.646319+0000","last_became_peered":"2024-05-08T06:36:40.646319+0000","last_unstale":"2024-05-08T06:36:43.109476+0000","last_undegraded":"2024-05-08T06:36:43.109476+0000","last_fullsized":"2024-05-08T06:36:43.109476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:03.574076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109441+0000","last_change":"2024-05-08T06:36:41.656383+0000","last_active":"2024-05-08T06:36:43.109441+0000","last_peered":"2024-05-08T06:36:43.109441+0000","last_clean":"2024-05-08T06:36:43.109441+0000","last_became_active":"2024-05-08T06:36:41.656059+0000","last_became_peered":"2024-05-08T06:36:41.656059+0000","last_unstale":"2024-05-08T06:36:43.109441+0000","last_undegraded":"2024-05-08T06:36:43.109441+0000","last_fullsized":"2024-05-08T06:36:43.109441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:49.740862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109265+0000","last_change":"2024-05-08T06:36:40.646560+0000","last_active":"2024-05-08T06:36:43.109265+0000","last_peered":"2024-05-08T06:36:43.109265+0000","last_clean":"2024-05-08T06:36:43.109265+0000","last_became_active":"2024-05-08T06:36:40.646285+0000","last_became_peered":"2024-05-08T06:36:40.646285+0000","last_unstale":"2024-05-08T06:36:43.109265+0000","last_undegraded":"2024-05-08T06:36:43.109265+0000","last_fullsized":"2024-05-08T06:36:43.109265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:56.400342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.108439+0000","last_change":"2024-05-08T06:36:41.661975+0000","last_active":"2024-05-08T06:36:43.108439+0000","last_peered":"2024-05-08T06:36:43.108439+0000","last_clean":"2024-05-08T06:36:43.108439+0000","last_became_active":"2024-05-08T06:36:41.661759+0000","last_became_peered":"2024-05-08T06:36:41.661759+0000","last_unstale":"2024-05-08T06:36:43.108439+0000","last_undegraded":"2024-05-08T06:36:43.108439+0000","last_fullsized":"2024-05-08T06:36:43.108439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:12.237215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109215+0000","last_change":"2024-05-08T06:36:40.645426+0000","last_active":"2024-05-08T06:36:43.109215+0000","last_peered":"2024-05-08T06:36:43.109215+0000","last_clean":"2024-05-08T06:36:43.109215+0000","last_became_active":"2024-05-08T06:36:40.645194+0000","last_became_peered":"2024-05-08T06:36:40.645194+0000","last_unstale":"2024-05-08T06:36:43.109215+0000","last_undegraded":"2024-05-08T06:36:43.109215+0000","last_fullsized":"2024-05-08T06:36:43.109215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109153+0000","last_change":"2024-05-08T06:36:41.660148+0000","last_active":"2024-05-08T06:36:43.109153+0000","last_peered":"2024-05-08T06:36:43.109153+0000","last_clean":"2024-05-08T06:36:43.109153+0000","last_became_active":"2024-05-08T06:36:41.659907+0000","last_became_peered":"2024-05-08T06:36:41.659907+0000","last_unstale":"2024-05-08T06:36:43.109153+0000","last_undegraded":"2024-05-08T06:36:43.109153+0000","last_fullsized":"2024-05-08T06:36:43.109153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:40.623604+0000","last_clean_scrub_stamp":"2024-05-08T06:36:40.623604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:45:07.961465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T06:36:43.109158+0000","last_change":"2024-05-08T06:36:40.645379+0000","last_active":"2024-05-08T06:36:43.109158+0000","last_peered":"2024-05-08T06:36:43.109158+0000","last_clean":"2024-05-08T06:36:43.109158+0000","last_became_active":"2024-05-08T06:36:40.645091+0000","last_became_peered":"2024-05-08T06:36:40.645091+0000","last_unstale":"2024-05-08T06:36:43.109158+0000","last_undegraded":"2024-05-08T06:36:43.109158+0000","last_fullsized":"2024-05-08T06:36:43.109158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:39.618012+0000","last_clean_scrub_stamp":"2024-05-08T06:36:39.618012+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:36.783850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:36:54.853 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T06:36:54.853 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T06:36:54.853 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-08T06:36:54.853 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T06:36:54.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.991+0000 7f0361c59640 1 Processor -- start 2024-05-08T06:36:54.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.991+0000 7f0361c59640 1 -- start start 2024-05-08T06:36:54.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.991+0000 7f0361c59640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f035c146560 0x7f035c0b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.991+0000 7f0361c59640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f035c0d2a30 0x7f035c15d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.991+0000 7f0361c59640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f035c072db0 con 0x7f035c0d1650 2024-05-08T06:36:54.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.991+0000 7f0361c59640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f035c072ad0 con 0x7f035c146560 2024-05-08T06:36:54.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.991+0000 7f0361c59640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f035c0b61d0 con 0x7f035c0d2a30 2024-05-08T06:36:54.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035bfff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f035c0d1650 0x7f035c0d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57210/0 (socket says 172.21.15.163:57210) 2024-05-08T06:36:54.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035bfff640 1 -- 172.21.15.163:0/2857383574 learned_addr learned my addr 172.21.15.163:0/2857383574 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:54.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035b7fe640 1 --2- 172.21.15.163:0/2857383574 >> v2:172.21.15.192:3300/0 conn(0x7f035c146560 0x7f035c0b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035affd640 1 --2- 172.21.15.163:0/2857383574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f035c0d2a30 0x7f035c15d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 -- 172.21.15.163:0/2857383574 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1237565567 0 0) 0x7f035c072db0 con 0x7f035c0d1650 2024-05-08T06:36:54.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 -- 172.21.15.163:0/2857383574 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0344003520 con 0x7f035c0d1650 2024-05-08T06:36:54.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 -- 172.21.15.163:0/2857383574 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3974316689 0 0) 0x7f0344003520 con 0x7f035c0d1650 2024-05-08T06:36:54.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 -- 172.21.15.163:0/2857383574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f035c0d2a30 msgr2=0x7f035c15d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:54.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 --2- 172.21.15.163:0/2857383574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f035c0d2a30 0x7f035c15d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 -- 172.21.15.163:0/2857383574 >> v2:172.21.15.192:3300/0 conn(0x7f035c146560 msgr2=0x7f035c0b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:54.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 --2- 172.21.15.163:0/2857383574 >> v2:172.21.15.192:3300/0 conn(0x7f035c146560 0x7f035c0b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 -- 172.21.15.163:0/2857383574 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035c0b64b0 con 0x7f035c0d1650 2024-05-08T06:36:54.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 -- 172.21.15.163:0/2857383574 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f0354001a70 con 0x7f035c0d1650 2024-05-08T06:36:54.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 -- 172.21.15.163:0/2857383574 >> v1:172.21.15.163:6789/0 conn(0x7f035c0d1650 legacy=0x7f035c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 --2- 172.21.15.163:0/2857383574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0344003a00 0x7f0344003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 --2- 172.21.15.163:0/2857383574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f035c0d2a30 0x7f0344004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 --2- 172.21.15.163:0/2857383574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0344004d40 0x7f03440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:54.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 -- 172.21.15.163:0/2857383574 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f035c072ad0 con 0x7f0344004d40 2024-05-08T06:36:54.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 -- 172.21.15.163:0/2857383574 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f035c0b61d0 con 0x7f0344003a00 2024-05-08T06:36:54.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035a7fc640 1 -- 172.21.15.163:0/2857383574 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f034400ba40 con 0x7f035c0d2a30 2024-05-08T06:36:54.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035bfff640 1 --2- 172.21.15.163:0/2857383574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0344004d40 0x7f03440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035b7fe640 1 --2- 172.21.15.163:0/2857383574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0344003a00 0x7f0344003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.992+0000 7f035affd640 1 --2- 172.21.15.163:0/2857383574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f035c0d2a30 0x7f0344004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:54.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.993+0000 7f035bfff640 1 -- 172.21.15.163:0/2857383574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f035c0d2a30 msgr2=0x7f0344004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:54.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.993+0000 7f035bfff640 1 --2- 172.21.15.163:0/2857383574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f035c0d2a30 0x7f0344004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.993+0000 7f035bfff640 1 -- 172.21.15.163:0/2857383574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0344003a00 msgr2=0x7f0344003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:54.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.993+0000 7f035bfff640 1 --2- 172.21.15.163:0/2857383574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0344003a00 0x7f0344003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:54.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.993+0000 7f035bfff640 1 -- 172.21.15.163:0/2857383574 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0344003520 con 0x7f0344004d40 2024-05-08T06:36:54.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.993+0000 7f035affd640 1 --2- 172.21.15.163:0/2857383574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f035c0d2a30 0x7f0344004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:36:54.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.993+0000 7f035bfff640 1 --2- 172.21.15.163:0/2857383574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0344004d40 0x7f03440051c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f035400a920 tx=0x7f0344005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd1fd03892caf32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:54.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.993+0000 7f035a7fc640 1 -- 172.21.15.163:0/2857383574 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035400a4c0 con 0x7f0344004d40 2024-05-08T06:36:54.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.993+0000 7f035a7fc640 1 -- 172.21.15.163:0/2857383574 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f035402f070 con 0x7f0344004d40 2024-05-08T06:36:54.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.993+0000 7f035a7fc640 1 -- 172.21.15.163:0/2857383574 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035402bc50 con 0x7f0344004d40 2024-05-08T06:36:54.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.994+0000 7f0361c59640 1 -- 172.21.15.163:0/2857383574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0344004d40 msgr2=0x7f03440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:54.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.994+0000 7f0361c59640 1 --2- 172.21.15.163:0/2857383574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0344004d40 0x7f03440051c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f035400a920 tx=0x7f0344005830 comp rx=0 tx=0).stop 2024-05-08T06:36:54.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.994+0000 7f035bfff640 1 -- 172.21.15.163:0/2857383574 reap_dead start 2024-05-08T06:36:54.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.994+0000 7f0361c59640 1 -- 172.21.15.163:0/2857383574 shutdown_connections 2024-05-08T06:36:54.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.994+0000 7f0361c59640 1 -- 172.21.15.163:0/2857383574 >> 172.21.15.163:0/2857383574 conn(0x7f035c0b4da0 msgr2=0x7f035c0b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:54.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.994+0000 7f0361c59640 1 -- 172.21.15.163:0/2857383574 shutdown_connections 2024-05-08T06:36:54.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.994+0000 7f0361c59640 1 -- 172.21.15.163:0/2857383574 wait complete. 2024-05-08T06:36:55.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.995+0000 7f0361c59640 1 Processor -- start 2024-05-08T06:36:55.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.995+0000 7f0361c59640 1 -- start start 2024-05-08T06:36:55.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f0361c59640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0344004d40 0x7f035c156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f0361c59640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f035c138ba0 0x7f035c13eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f0361c59640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f035c13eff0 0x7f035c13c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f0361c59640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f035c15d890 con 0x7f0344004d40 2024-05-08T06:36:55.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f0361c59640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f035c146940 con 0x7f035c138ba0 2024-05-08T06:36:55.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f0361c59640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f035c15da10 con 0x7f035c13eff0 2024-05-08T06:36:55.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f035b7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0344004d40 0x7f035c156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f035b7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0344004d40 0x7f035c156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44806/0 (socket says 172.21.15.163:44806) 2024-05-08T06:36:55.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f035b7fe640 1 -- 172.21.15.163:0/1622224231 learned_addr learned my addr 172.21.15.163:0/1622224231 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:55.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f035affd640 1 --2- 172.21.15.163:0/1622224231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f035c138ba0 0x7f035c13eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f035bfff640 1 --2- 172.21.15.163:0/1622224231 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f035c13eff0 0x7f035c13c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f035b7fe640 1 -- 172.21.15.163:0/1622224231 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f035c13eff0 msgr2=0x7f035c13c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f035b7fe640 1 --2- 172.21.15.163:0/1622224231 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f035c13eff0 0x7f035c13c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f035b7fe640 1 -- 172.21.15.163:0/1622224231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f035c138ba0 msgr2=0x7f035c13eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.996+0000 7f035b7fe640 1 --2- 172.21.15.163:0/1622224231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f035c138ba0 0x7f035c13eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.997+0000 7f035b7fe640 1 -- 172.21.15.163:0/1622224231 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035c1393f0 con 0x7f0344004d40 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.997+0000 7f035b7fe640 1 --2- 172.21.15.163:0/1622224231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0344004d40 0x7f035c156590 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f034c011960 tx=0x7f034c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.997+0000 7f0343fff640 1 -- 172.21.15.163:0/1622224231 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c00faf0 con 0x7f0344004d40 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.997+0000 7f0343fff640 1 -- 172.21.15.163:0/1622224231 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034c00fc90 con 0x7f0344004d40 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.997+0000 7f0361c59640 1 -- 172.21.15.163:0/1622224231 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035c1396d0 con 0x7f0344004d40 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.997+0000 7f0343fff640 1 -- 172.21.15.163:0/1622224231 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c014440 con 0x7f0344004d40 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.997+0000 7f0361c59640 1 -- 172.21.15.163:0/1622224231 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035c13c1c0 con 0x7f0344004d40 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.998+0000 7f0361c59640 1 -- 172.21.15.163:0/1622224231 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035c13f400 con 0x7f0344004d40 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.998+0000 7f0343fff640 1 -- 172.21.15.163:0/1622224231 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f034c012040 con 0x7f0344004d40 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.999+0000 7f0343fff640 1 --2- 172.21.15.163:0/1622224231 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0348076300 0x7f03480787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.999+0000 7f0343fff640 1 -- 172.21.15.163:0/1622224231 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f034c097c60 con 0x7f0344004d40 2024-05-08T06:36:55.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.999+0000 7f035affd640 1 --2- 172.21.15.163:0/1622224231 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0348076300 0x7f03480787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:54.999+0000 7f035affd640 1 --2- 172.21.15.163:0/1622224231 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0348076300 0x7f03480787c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f035c145cb0 tx=0x7f0350008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:55.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.003+0000 7f0343fff640 1 -- 172.21.15.163:0/1622224231 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f035c13f400 con 0x7f0344004d40 2024-05-08T06:36:55.220 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.221+0000 7f0361c59640 1 -- 172.21.15.163:0/1622224231 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f035c06a4b0 con 0x7f0344004d40 2024-05-08T06:36:55.220 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.221+0000 7f0343fff640 1 -- 172.21.15.163:0/1622224231 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f034c019030 con 0x7f0344004d40 2024-05-08T06:36:55.220 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:55.221 INFO:teuthology.orchestra.run.smithi163.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T06:36:55.222 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.223+0000 7f0361c59640 1 -- 172.21.15.163:0/1622224231 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0348076300 msgr2=0x7f03480787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.222 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.223+0000 7f0361c59640 1 --2- 172.21.15.163:0/1622224231 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0348076300 0x7f03480787c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f035c145cb0 tx=0x7f0350008040 comp rx=0 tx=0).stop 2024-05-08T06:36:55.222 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.223+0000 7f0361c59640 1 -- 172.21.15.163:0/1622224231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0344004d40 msgr2=0x7f035c156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.223+0000 7f0361c59640 1 --2- 172.21.15.163:0/1622224231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0344004d40 0x7f035c156590 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f034c011960 tx=0x7f034c011e20 comp rx=0 tx=0).stop 2024-05-08T06:36:55.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.223+0000 7f0361c59640 1 -- 172.21.15.163:0/1622224231 shutdown_connections 2024-05-08T06:36:55.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.223+0000 7f0361c59640 1 --2- 172.21.15.163:0/1622224231 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0348076300 0x7f03480787c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.223+0000 7f0361c59640 1 --2- 172.21.15.163:0/1622224231 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f035c13eff0 0x7f035c13c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.223+0000 7f0361c59640 1 --2- 172.21.15.163:0/1622224231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f035c138ba0 0x7f035c13eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.223+0000 7f0361c59640 1 --2- 172.21.15.163:0/1622224231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0344004d40 0x7f035c156590 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.223+0000 7f0361c59640 1 -- 172.21.15.163:0/1622224231 >> 172.21.15.163:0/1622224231 conn(0x7f035c0b4da0 msgr2=0x7f035c143170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:55.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.223+0000 7f0361c59640 1 -- 172.21.15.163:0/1622224231 shutdown_connections 2024-05-08T06:36:55.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.223+0000 7f0361c59640 1 -- 172.21.15.163:0/1622224231 wait complete. 2024-05-08T06:36:55.233 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-08T06:36:55.233 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:36:55.334 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.335+0000 7f8d46fda640 1 Processor -- start 2024-05-08T06:36:55.334 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.335+0000 7f8d46fda640 1 -- start start 2024-05-08T06:36:55.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.335+0000 7f8d46fda640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40140070 0x7f8d40071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.335+0000 7f8d46fda640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8d40142f20 0x7f8d401410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.335+0000 7f8d46fda640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d40140450 con 0x7f8d400722f0 2024-05-08T06:36:55.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.335+0000 7f8d46fda640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f8d40072e50 con 0x7f8d40142f20 2024-05-08T06:36:55.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.335+0000 7f8d46fda640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8d40072890 con 0x7f8d40140070 2024-05-08T06:36:55.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.336+0000 7f8d45577640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8d40142f20 0x7f8d401410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.336+0000 7f8d3ffff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8d400722f0 0x7f8d40142810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57212/0 (socket says 172.21.15.163:57212) 2024-05-08T06:36:55.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.336+0000 7f8d3ffff640 1 -- 172.21.15.163:0/3788826936 learned_addr learned my addr 172.21.15.163:0/3788826936 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:55.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.336+0000 7f8d44d76640 1 --2- 172.21.15.163:0/3788826936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40140070 0x7f8d40071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.336+0000 7f8d45577640 1 -- 172.21.15.163:0/3788826936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40140070 msgr2=0x7f8d40071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:55.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.336+0000 7f8d45577640 1 --2- 172.21.15.163:0/3788826936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40140070 0x7f8d40071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.336+0000 7f8d45577640 1 -- 172.21.15.163:0/3788826936 >> v1:172.21.15.163:6789/0 conn(0x7f8d400722f0 legacy=0x7f8d40142810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.336+0000 7f8d45577640 1 -- 172.21.15.163:0/3788826936 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d40072b70 con 0x7f8d40142f20 2024-05-08T06:36:55.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.336+0000 7f8d3f7fe640 1 -- 172.21.15.163:0/3788826936 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2518250518 0 0) 0x7f8d40140450 con 0x7f8d400722f0 2024-05-08T06:36:55.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.337+0000 7f8d45577640 1 --2- 172.21.15.163:0/3788826936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d40142f20 0x7f8d401410b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8d380097a0 tx=0x7f8d3802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcfb7d8c9218cbc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:55.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.337+0000 7f8d3f7fe640 1 -- 172.21.15.163:0/3788826936 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38037070 con 0x7f8d40142f20 2024-05-08T06:36:55.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.337+0000 7f8d3f7fe640 1 -- 172.21.15.163:0/3788826936 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d3802adb0 con 0x7f8d40142f20 2024-05-08T06:36:55.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.337+0000 7f8d3f7fe640 1 -- 172.21.15.163:0/3788826936 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38032660 con 0x7f8d40142f20 2024-05-08T06:36:55.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.337+0000 7f8d46fda640 1 -- 172.21.15.163:0/3788826936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d40142f20 msgr2=0x7f8d401410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.337+0000 7f8d46fda640 1 --2- 172.21.15.163:0/3788826936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d40142f20 0x7f8d401410b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8d380097a0 tx=0x7f8d3802a060 comp rx=0 tx=0).stop 2024-05-08T06:36:55.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.337+0000 7f8d46fda640 1 -- 172.21.15.163:0/3788826936 shutdown_connections 2024-05-08T06:36:55.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.337+0000 7f8d46fda640 1 --2- 172.21.15.163:0/3788826936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d40142f20 0x7f8d401410b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.337+0000 7f8d46fda640 1 --2- 172.21.15.163:0/3788826936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40140070 0x7f8d40071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.337+0000 7f8d46fda640 1 -- 172.21.15.163:0/3788826936 >> 172.21.15.163:0/3788826936 conn(0x7f8d400b4da0 msgr2=0x7f8d400b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:55.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.337+0000 7f8d46fda640 1 -- 172.21.15.163:0/3788826936 shutdown_connections 2024-05-08T06:36:55.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.337+0000 7f8d46fda640 1 -- 172.21.15.163:0/3788826936 wait complete. 2024-05-08T06:36:55.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.338+0000 7f8d46fda640 1 Processor -- start 2024-05-08T06:36:55.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.338+0000 7f8d46fda640 1 -- start start 2024-05-08T06:36:55.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.339+0000 7f8d46fda640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d400722f0 0x7f8d4015f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.339+0000 7f8d46fda640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40140070 0x7f8d4015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.339+0000 7f8d46fda640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d40142f20 0x7f8d4015c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.339+0000 7f8d46fda640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8d401415f0 con 0x7f8d40142f20 2024-05-08T06:36:55.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.339+0000 7f8d46fda640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8d401435b0 con 0x7f8d400722f0 2024-05-08T06:36:55.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.339+0000 7f8d46fda640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8d40141770 con 0x7f8d40140070 2024-05-08T06:36:55.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d45577640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d40142f20 0x7f8d4015c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d45577640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d40142f20 0x7f8d4015c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44824/0 (socket says 172.21.15.163:44824) 2024-05-08T06:36:55.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d45577640 1 -- 172.21.15.163:0/2420712222 learned_addr learned my addr 172.21.15.163:0/2420712222 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:55.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d3ffff640 1 --2- 172.21.15.163:0/2420712222 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40140070 0x7f8d4015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d44d76640 1 --2- 172.21.15.163:0/2420712222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d400722f0 0x7f8d4015f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d45577640 1 -- 172.21.15.163:0/2420712222 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40140070 msgr2=0x7f8d4015c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d45577640 1 --2- 172.21.15.163:0/2420712222 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40140070 0x7f8d4015c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d45577640 1 -- 172.21.15.163:0/2420712222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d400722f0 msgr2=0x7f8d4015f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d45577640 1 --2- 172.21.15.163:0/2420712222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d400722f0 0x7f8d4015f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d45577640 1 -- 172.21.15.163:0/2420712222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d40160910 con 0x7f8d40142f20 2024-05-08T06:36:55.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d45577640 1 --2- 172.21.15.163:0/2420712222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d40142f20 0x7f8d4015c700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d380098c0 tx=0x7f8d380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:55.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d3d7fa640 1 -- 172.21.15.163:0/2420712222 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38041020 con 0x7f8d40142f20 2024-05-08T06:36:55.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d3d7fa640 1 -- 172.21.15.163:0/2420712222 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d3803c670 con 0x7f8d40142f20 2024-05-08T06:36:55.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d46fda640 1 -- 172.21.15.163:0/2420712222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d40160bf0 con 0x7f8d40142f20 2024-05-08T06:36:55.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d3d7fa640 1 -- 172.21.15.163:0/2420712222 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38037040 con 0x7f8d40142f20 2024-05-08T06:36:55.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.340+0000 7f8d46fda640 1 -- 172.21.15.163:0/2420712222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d401611b0 con 0x7f8d40142f20 2024-05-08T06:36:55.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.341+0000 7f8d46fda640 1 -- 172.21.15.163:0/2420712222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d4006a550 con 0x7f8d40142f20 2024-05-08T06:36:55.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.342+0000 7f8d3d7fa640 1 -- 172.21.15.163:0/2420712222 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8d38044020 con 0x7f8d40142f20 2024-05-08T06:36:55.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.342+0000 7f8d3d7fa640 1 --2- 172.21.15.163:0/2420712222 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8d2c0762d0 0x7f8d2c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.342+0000 7f8d44d76640 1 --2- 172.21.15.163:0/2420712222 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8d2c0762d0 0x7f8d2c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.342+0000 7f8d3d7fa640 1 -- 172.21.15.163:0/2420712222 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8d380b7bd0 con 0x7f8d40142f20 2024-05-08T06:36:55.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.343+0000 7f8d44d76640 1 --2- 172.21.15.163:0/2420712222 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8d2c0762d0 0x7f8d2c078790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8d30006fd0 tx=0x7f8d30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:55.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.346+0000 7f8d3d7fa640 1 -- 172.21.15.163:0/2420712222 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8d3804b030 con 0x7f8d40142f20 2024-05-08T06:36:55.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.564+0000 7f8d46fda640 1 -- 172.21.15.163:0/2420712222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d400b5a40 con 0x7f8d40142f20 2024-05-08T06:36:55.564 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.565+0000 7f8d3d7fa640 1 -- 172.21.15.163:0/2420712222 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f8d38080ee0 con 0x7f8d40142f20 2024-05-08T06:36:55.564 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:55.564 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2413407842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2413407842},{"type":"v1","addr":"172.21.15.163:6835","nonce":2413407842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2405203133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2405203133},{"type":"v1","addr":"172.21.15.163:6837","nonce":2405203133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3301087897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3301087897},{"type":"v1","addr":"172.21.15.192:6833","nonce":3301087897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:36:40.985978+0000","modified":"2024-05-08T06:36:43.900778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/354782184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":354782184},{"type":"v1","addr":"172.21.15.192:6835","nonce":354782184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-05-08T06:36:55.564 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 7 2024-05-08T06:36:55.566 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.567+0000 7f8d46fda640 1 -- 172.21.15.163:0/2420712222 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8d2c0762d0 msgr2=0x7f8d2c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.566 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.567+0000 7f8d46fda640 1 --2- 172.21.15.163:0/2420712222 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8d2c0762d0 0x7f8d2c078790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8d30006fd0 tx=0x7f8d30008040 comp rx=0 tx=0).stop 2024-05-08T06:36:55.566 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.567+0000 7f8d46fda640 1 -- 172.21.15.163:0/2420712222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d40142f20 msgr2=0x7f8d4015c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.566 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.567+0000 7f8d46fda640 1 --2- 172.21.15.163:0/2420712222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d40142f20 0x7f8d4015c700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d380098c0 tx=0x7f8d380046c0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.566 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.567+0000 7f8d46fda640 1 -- 172.21.15.163:0/2420712222 shutdown_connections 2024-05-08T06:36:55.567 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.567+0000 7f8d46fda640 1 --2- 172.21.15.163:0/2420712222 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8d2c0762d0 0x7f8d2c078790 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.567 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.567+0000 7f8d46fda640 1 --2- 172.21.15.163:0/2420712222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d40142f20 0x7f8d4015c700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.567 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.567+0000 7f8d46fda640 1 --2- 172.21.15.163:0/2420712222 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40140070 0x7f8d4015c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.567 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.567+0000 7f8d46fda640 1 --2- 172.21.15.163:0/2420712222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d400722f0 0x7f8d4015f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.567 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.567+0000 7f8d46fda640 1 -- 172.21.15.163:0/2420712222 >> 172.21.15.163:0/2420712222 conn(0x7f8d400b4da0 msgr2=0x7f8d40141b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:55.567 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.567+0000 7f8d46fda640 1 -- 172.21.15.163:0/2420712222 shutdown_connections 2024-05-08T06:36:55.567 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.567+0000 7f8d46fda640 1 -- 172.21.15.163:0/2420712222 wait complete. 2024-05-08T06:36:55.578 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:36:55.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.678+0000 7f58e1493640 1 Processor -- start 2024-05-08T06:36:55.678 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.678+0000 7f58e1493640 1 -- start start 2024-05-08T06:36:55.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.679+0000 7f58e1493640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f58dc0d0a00 0x7f58dc0b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.679+0000 7f58e1493640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58dc0d0080 con 0x7f58dc0cfc10 2024-05-08T06:36:55.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.679+0000 7f58e1493640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58dc0d36f0 con 0x7f58dc15f760 2024-05-08T06:36:55.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.679+0000 7f58e1493640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f58dc15fb40 con 0x7f58dc0d0a00 2024-05-08T06:36:55.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.679+0000 7f58d27fc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f58dc0d0a00 0x7f58dc0b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.679+0000 7f58d27fc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f58dc0d0a00 0x7f58dc0b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44832/0 (socket says 172.21.15.163:44832) 2024-05-08T06:36:55.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.679+0000 7f58d27fc640 1 -- 172.21.15.163:0/3667159458 learned_addr learned my addr 172.21.15.163:0/3667159458 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:55.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.679+0000 7f58db7fe640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f58dc0cfc10 0x7f58dc0b8710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57214/0 (socket says 172.21.15.163:57214) 2024-05-08T06:36:55.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.680+0000 7f58d27fc640 1 -- 172.21.15.163:0/3667159458 >> v1:172.21.15.192:6789/0 conn(0x7f58dc15f760 legacy=0x7f58dc0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.680+0000 7f58d27fc640 1 -- 172.21.15.163:0/3667159458 >> v1:172.21.15.163:6789/0 conn(0x7f58dc0cfc10 legacy=0x7f58dc0b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.679 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.680+0000 7f58d27fc640 1 -- 172.21.15.163:0/3667159458 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc0d39d0 con 0x7f58dc0d0a00 2024-05-08T06:36:55.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.680+0000 7f58da7fc640 1 -- 172.21.15.163:0/3667159458 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1031972725 0 0) 0x7f58dc0d0080 con 0x7f58dc0cfc10 2024-05-08T06:36:55.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.680+0000 7f58d27fc640 1 --2- 172.21.15.163:0/3667159458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc0d0a00 0x7f58dc0b4770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009870 tx=0x7f58c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15148c9c1be4d05e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:55.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.680+0000 7f58da7fc640 1 -- 172.21.15.163:0/3667159458 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4037070 con 0x7f58dc0d0a00 2024-05-08T06:36:55.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.680+0000 7f58da7fc640 1 -- 172.21.15.163:0/3667159458 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c4032440 con 0x7f58dc0d0a00 2024-05-08T06:36:55.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.680+0000 7f58da7fc640 1 -- 172.21.15.163:0/3667159458 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4032810 con 0x7f58dc0d0a00 2024-05-08T06:36:55.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.681+0000 7f58e1493640 1 -- 172.21.15.163:0/3667159458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc0d0a00 msgr2=0x7f58dc0b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.681+0000 7f58e1493640 1 --2- 172.21.15.163:0/3667159458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc0d0a00 0x7f58dc0b4770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009870 tx=0x7f58c402a270 comp rx=0 tx=0).stop 2024-05-08T06:36:55.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.681+0000 7f58e1493640 1 -- 172.21.15.163:0/3667159458 shutdown_connections 2024-05-08T06:36:55.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.681+0000 7f58e1493640 1 --2- 172.21.15.163:0/3667159458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc0d0a00 0x7f58dc0b4770 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.681+0000 7f58e1493640 1 -- 172.21.15.163:0/3667159458 >> 172.21.15.163:0/3667159458 conn(0x7f58dc0740e0 msgr2=0x7f58dc06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:55.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.681+0000 7f58e1493640 1 -- 172.21.15.163:0/3667159458 shutdown_connections 2024-05-08T06:36:55.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.681+0000 7f58e1493640 1 -- 172.21.15.163:0/3667159458 wait complete. 2024-05-08T06:36:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.682+0000 7f58e1493640 1 Processor -- start 2024-05-08T06:36:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.682+0000 7f58e1493640 1 -- start start 2024-05-08T06:36:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58e1493640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58dc0cfc10 0x7f58dc157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58e1493640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc0d0a00 0x7f58dc0cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58e1493640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58dc15f760 0x7f58dc156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58e1493640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc0d4270 con 0x7f58dc0d0a00 2024-05-08T06:36:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58e1493640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc0d3f90 con 0x7f58dc0cfc10 2024-05-08T06:36:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58e1493640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f58dc0d4790 con 0x7f58dc15f760 2024-05-08T06:36:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58d27fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc0d0a00 0x7f58dc0cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58d27fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc0d0a00 0x7f58dc0cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44842/0 (socket says 172.21.15.163:44842) 2024-05-08T06:36:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58d27fc640 1 -- 172.21.15.163:0/1531087695 learned_addr learned my addr 172.21.15.163:0/1531087695 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58db7fe640 1 --2- 172.21.15.163:0/1531087695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58dc15f760 0x7f58dc156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58daffd640 1 --2- 172.21.15.163:0/1531087695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58dc0cfc10 0x7f58dc157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58d27fc640 1 -- 172.21.15.163:0/1531087695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58dc15f760 msgr2=0x7f58dc156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58d27fc640 1 --2- 172.21.15.163:0/1531087695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58dc15f760 0x7f58dc156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58d27fc640 1 -- 172.21.15.163:0/1531087695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58dc0cfc10 msgr2=0x7f58dc157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.683+0000 7f58d27fc640 1 --2- 172.21.15.163:0/1531087695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58dc0cfc10 0x7f58dc157060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.684+0000 7f58d27fc640 1 -- 172.21.15.163:0/1531087695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc0b58f0 con 0x7f58dc0d0a00 2024-05-08T06:36:55.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.684+0000 7f58d27fc640 1 --2- 172.21.15.163:0/1531087695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc0d0a00 0x7f58dc0cb8a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009870 tx=0x7f58c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:55.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.684+0000 7f58d3fff640 1 -- 172.21.15.163:0/1531087695 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4041020 con 0x7f58dc0d0a00 2024-05-08T06:36:55.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.684+0000 7f58d3fff640 1 -- 172.21.15.163:0/1531087695 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c4030970 con 0x7f58dc0d0a00 2024-05-08T06:36:55.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.684+0000 7f58d3fff640 1 -- 172.21.15.163:0/1531087695 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4037040 con 0x7f58dc0d0a00 2024-05-08T06:36:55.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.684+0000 7f58e1493640 1 -- 172.21.15.163:0/1531087695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc0b5bd0 con 0x7f58dc0d0a00 2024-05-08T06:36:55.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.684+0000 7f58e1493640 1 -- 172.21.15.163:0/1531087695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc0b6190 con 0x7f58dc0d0a00 2024-05-08T06:36:55.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.685+0000 7f58e1493640 1 -- 172.21.15.163:0/1531087695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58dc06a550 con 0x7f58dc0d0a00 2024-05-08T06:36:55.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.685+0000 7f58d3fff640 1 -- 172.21.15.163:0/1531087695 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f58c4044020 con 0x7f58dc0d0a00 2024-05-08T06:36:55.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.686+0000 7f58d3fff640 1 --2- 172.21.15.163:0/1531087695 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f58bc0762d0 0x7f58bc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:55.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.686+0000 7f58d3fff640 1 -- 172.21.15.163:0/1531087695 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f58c40b7f50 con 0x7f58dc0d0a00 2024-05-08T06:36:55.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.686+0000 7f58daffd640 1 --2- 172.21.15.163:0/1531087695 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f58bc0762d0 0x7f58bc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:55.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.686+0000 7f58daffd640 1 --2- 172.21.15.163:0/1531087695 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f58bc0762d0 0x7f58bc078790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f58cc000f60 tx=0x7f58cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:55.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.690+0000 7f58d3fff640 1 -- 172.21.15.163:0/1531087695 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f58c404b030 con 0x7f58dc0d0a00 2024-05-08T06:36:55.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.910+0000 7f58e1493640 1 -- 172.21.15.163:0/1531087695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58dc0d4e00 con 0x7f58dc0d0a00 2024-05-08T06:36:55.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.910+0000 7f58d3fff640 1 -- 172.21.15.163:0/1531087695 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f58c4081940 con 0x7f58dc0d0a00 2024-05-08T06:36:55.910 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:55.910 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2413407842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2413407842},{"type":"v1","addr":"172.21.15.163:6835","nonce":2413407842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2405203133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2405203133},{"type":"v1","addr":"172.21.15.163:6837","nonce":2405203133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3301087897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3301087897},{"type":"v1","addr":"172.21.15.192:6833","nonce":3301087897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:36:40.985978+0000","modified":"2024-05-08T06:36:43.900778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/354782184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":354782184},{"type":"v1","addr":"172.21.15.192:6835","nonce":354782184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-05-08T06:36:55.910 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 7 2024-05-08T06:36:55.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.912+0000 7f58e1493640 1 -- 172.21.15.163:0/1531087695 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f58bc0762d0 msgr2=0x7f58bc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.912+0000 7f58e1493640 1 --2- 172.21.15.163:0/1531087695 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f58bc0762d0 0x7f58bc078790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f58cc000f60 tx=0x7f58cc009040 comp rx=0 tx=0).stop 2024-05-08T06:36:55.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.912+0000 7f58e1493640 1 -- 172.21.15.163:0/1531087695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc0d0a00 msgr2=0x7f58dc0cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:55.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.912+0000 7f58e1493640 1 --2- 172.21.15.163:0/1531087695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc0d0a00 0x7f58dc0cb8a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009870 tx=0x7f58c40046c0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.912+0000 7f58e1493640 1 -- 172.21.15.163:0/1531087695 shutdown_connections 2024-05-08T06:36:55.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.912+0000 7f58e1493640 1 --2- 172.21.15.163:0/1531087695 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f58bc0762d0 0x7f58bc078790 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.912+0000 7f58e1493640 1 --2- 172.21.15.163:0/1531087695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58dc15f760 0x7f58dc156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.912+0000 7f58e1493640 1 --2- 172.21.15.163:0/1531087695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc0d0a00 0x7f58dc0cb8a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.912+0000 7f58e1493640 1 --2- 172.21.15.163:0/1531087695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58dc0cfc10 0x7f58dc157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:55.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.912+0000 7f58e1493640 1 -- 172.21.15.163:0/1531087695 >> 172.21.15.163:0/1531087695 conn(0x7f58dc0740e0 msgr2=0x7f58dc0ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:55.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.912+0000 7f58e1493640 1 -- 172.21.15.163:0/1531087695 shutdown_connections 2024-05-08T06:36:55.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:55.912+0000 7f58e1493640 1 -- 172.21.15.163:0/1531087695 wait complete. 2024-05-08T06:36:55.923 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:36:56.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.023+0000 7f2a23e37640 1 Processor -- start 2024-05-08T06:36:56.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.023+0000 7f2a23e37640 1 -- start start 2024-05-08T06:36:56.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.024+0000 7f2a23e37640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2a1c15d130 0x7f2a1c0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:56.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.024+0000 7f2a23e37640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a1c074830 con 0x7f2a1c0af310 2024-05-08T06:36:56.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.024+0000 7f2a23e37640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a1c073f90 con 0x7f2a1c0ae180 2024-05-08T06:36:56.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.024+0000 7f2a23e37640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f2a1c074270 con 0x7f2a1c15d130 2024-05-08T06:36:56.024 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.024+0000 7f2a21bd3640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2a1c15d130 0x7f2a1c0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:56.024 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.024+0000 7f2a21bd3640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2a1c15d130 0x7f2a1c0d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44858/0 (socket says 172.21.15.163:44858) 2024-05-08T06:36:56.024 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.024+0000 7f2a21bd3640 1 -- 172.21.15.163:0/1598638065 learned_addr learned my addr 172.21.15.163:0/1598638065 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:56.024 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.024+0000 7f2a21bd3640 1 -- 172.21.15.163:0/1598638065 >> v1:172.21.15.192:6789/0 conn(0x7f2a1c0ae180 legacy=0x7f2a1c0aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:56.024 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.024+0000 7f2a21bd3640 1 -- 172.21.15.163:0/1598638065 >> v1:172.21.15.163:6789/0 conn(0x7f2a1c0af310 legacy=0x7f2a1c06edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:56.024 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.024+0000 7f2a20bd1640 1 -- 172.21.15.163:0/1598638065 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 247418091 0 0) 0x7f2a1c074830 con 0x7f2a1c0af310 2024-05-08T06:36:56.024 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.024+0000 7f2a21bd3640 1 -- 172.21.15.163:0/1598638065 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1c074550 con 0x7f2a1c15d130 2024-05-08T06:36:56.025 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.025+0000 7f2a21bd3640 1 --2- 172.21.15.163:0/1598638065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a1c15d130 0x7f2a1c0d0720 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009870 tx=0x7f2a1802a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6c91452fb8fed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:56.025 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.025+0000 7f2a20bd1640 1 -- 172.21.15.163:0/1598638065 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18037070 con 0x7f2a1c15d130 2024-05-08T06:36:56.025 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.025+0000 7f2a20bd1640 1 -- 172.21.15.163:0/1598638065 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a18032640 con 0x7f2a1c15d130 2024-05-08T06:36:56.025 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.025+0000 7f2a20bd1640 1 -- 172.21.15.163:0/1598638065 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18032a10 con 0x7f2a1c15d130 2024-05-08T06:36:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.026+0000 7f2a23e37640 1 -- 172.21.15.163:0/1598638065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a1c15d130 msgr2=0x7f2a1c0d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.026+0000 7f2a23e37640 1 --2- 172.21.15.163:0/1598638065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a1c15d130 0x7f2a1c0d0720 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009870 tx=0x7f2a1802a3c0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.026+0000 7f2a23e37640 1 -- 172.21.15.163:0/1598638065 shutdown_connections 2024-05-08T06:36:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.026+0000 7f2a23e37640 1 --2- 172.21.15.163:0/1598638065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a1c15d130 0x7f2a1c0d0720 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.026+0000 7f2a23e37640 1 -- 172.21.15.163:0/1598638065 >> 172.21.15.163:0/1598638065 conn(0x7f2a1c06c500 msgr2=0x7f2a1c06c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.026+0000 7f2a23e37640 1 -- 172.21.15.163:0/1598638065 shutdown_connections 2024-05-08T06:36:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.026+0000 7f2a23e37640 1 -- 172.21.15.163:0/1598638065 wait complete. 2024-05-08T06:36:56.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.027+0000 7f2a23e37640 1 Processor -- start 2024-05-08T06:36:56.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.027+0000 7f2a23e37640 1 -- start start 2024-05-08T06:36:56.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a23e37640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a1c0ae180 0x7f2a1c15f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:56.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a23e37640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a1c0af310 0x7f2a1c1e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:56.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a23e37640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a1c15d130 0x7f2a1c1e6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:56.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a23e37640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c06f650 con 0x7f2a1c0ae180 2024-05-08T06:36:56.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a23e37640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c06f4d0 con 0x7f2a1c15d130 2024-05-08T06:36:56.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a23e37640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2a1c06f7d0 con 0x7f2a1c0af310 2024-05-08T06:36:56.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a21bd3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a1c0ae180 0x7f2a1c15f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:56.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a21bd3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a1c0ae180 0x7f2a1c15f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44864/0 (socket says 172.21.15.163:44864) 2024-05-08T06:36:56.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a21bd3640 1 -- 172.21.15.163:0/2447496767 learned_addr learned my addr 172.21.15.163:0/2447496767 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:56.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a223d4640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a1c15d130 0x7f2a1c1e6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:56.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a21bd3640 1 -- 172.21.15.163:0/2447496767 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a1c0af310 msgr2=0x7f2a1c1e69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:56.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a213d2640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a1c0af310 0x7f2a1c1e69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:56.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a21bd3640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a1c0af310 0x7f2a1c1e69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a21bd3640 1 -- 172.21.15.163:0/2447496767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a1c15d130 msgr2=0x7f2a1c1e6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a21bd3640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a1c15d130 0x7f2a1c1e6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a21bd3640 1 -- 172.21.15.163:0/2447496767 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1c15fc70 con 0x7f2a1c0ae180 2024-05-08T06:36:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a223d4640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a1c15d130 0x7f2a1c1e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a213d2640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a1c0af310 0x7f2a1c1e69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a21bd3640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a1c0ae180 0x7f2a1c15f730 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009870 tx=0x7f2a180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a127fc640 1 -- 172.21.15.163:0/2447496767 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18041020 con 0x7f2a1c0ae180 2024-05-08T06:36:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a127fc640 1 -- 172.21.15.163:0/2447496767 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a18030990 con 0x7f2a1c0ae180 2024-05-08T06:36:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a127fc640 1 -- 172.21.15.163:0/2447496767 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18037040 con 0x7f2a1c0ae180 2024-05-08T06:36:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a23e37640 1 -- 172.21.15.163:0/2447496767 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1c159f40 con 0x7f2a1c0ae180 2024-05-08T06:36:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.028+0000 7f2a23e37640 1 -- 172.21.15.163:0/2447496767 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1c15a500 con 0x7f2a1c0ae180 2024-05-08T06:36:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.029+0000 7f2a23e37640 1 -- 172.21.15.163:0/2447496767 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a1c06a550 con 0x7f2a1c0ae180 2024-05-08T06:36:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.030+0000 7f2a127fc640 1 -- 172.21.15.163:0/2447496767 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2a18044020 con 0x7f2a1c0ae180 2024-05-08T06:36:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.030+0000 7f2a127fc640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2a000762d0 0x7f2a00078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.030+0000 7f2a213d2640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2a000762d0 0x7f2a00078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.031+0000 7f2a127fc640 1 -- 172.21.15.163:0/2447496767 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2a180b7fe0 con 0x7f2a1c0ae180 2024-05-08T06:36:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.031+0000 7f2a213d2640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2a000762d0 0x7f2a00078790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0b1d50 tx=0x7f2a0c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:56.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.035+0000 7f2a127fc640 1 -- 172.21.15.163:0/2447496767 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2a1804b030 con 0x7f2a1c0ae180 2024-05-08T06:36:56.222 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.223+0000 7f2a23e37640 1 -- 172.21.15.163:0/2447496767 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a1c070800 con 0x7f2a1c0ae180 2024-05-08T06:36:56.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.224+0000 7f2a127fc640 1 -- 172.21.15.163:0/2447496767 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f2a18081940 con 0x7f2a1c0ae180 2024-05-08T06:36:56.224 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:56.224 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:36:42.641260+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:36:38.853461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:36:39.952926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:36:56.226 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.226+0000 7f2a23e37640 1 -- 172.21.15.163:0/2447496767 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2a000762d0 msgr2=0x7f2a00078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:56.226 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.226+0000 7f2a23e37640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2a000762d0 0x7f2a00078790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0b1d50 tx=0x7f2a0c009040 comp rx=0 tx=0).stop 2024-05-08T06:36:56.226 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.226+0000 7f2a23e37640 1 -- 172.21.15.163:0/2447496767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a1c0ae180 msgr2=0x7f2a1c15f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:56.226 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.226+0000 7f2a23e37640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a1c0ae180 0x7f2a1c15f730 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009870 tx=0x7f2a180046c0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.226 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.226+0000 7f2a23e37640 1 -- 172.21.15.163:0/2447496767 shutdown_connections 2024-05-08T06:36:56.226 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.226+0000 7f2a23e37640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2a000762d0 0x7f2a00078790 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.226 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.226+0000 7f2a23e37640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a1c15d130 0x7f2a1c1e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.226 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.226+0000 7f2a23e37640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a1c0af310 0x7f2a1c1e69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.226 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.226+0000 7f2a23e37640 1 --2- 172.21.15.163:0/2447496767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a1c0ae180 0x7f2a1c15f730 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.226 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.226+0000 7f2a23e37640 1 -- 172.21.15.163:0/2447496767 >> 172.21.15.163:0/2447496767 conn(0x7f2a1c06c500 msgr2=0x7f2a1c06fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:56.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.226+0000 7f2a23e37640 1 -- 172.21.15.163:0/2447496767 shutdown_connections 2024-05-08T06:36:56.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.226+0000 7f2a23e37640 1 -- 172.21.15.163:0/2447496767 wait complete. 2024-05-08T06:36:56.237 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:36:56.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.338+0000 7f56d31e4640 1 Processor -- start 2024-05-08T06:36:56.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.338+0000 7f56d31e4640 1 -- start start 2024-05-08T06:36:56.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.339+0000 7f56d31e4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f56cc15f760 0x7f56cc0b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:56.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.339+0000 7f56d31e4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56cc0d0a00 0x7f56cc0b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:56.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.339+0000 7f56d31e4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56cc0d0070 con 0x7f56cc0cfc10 2024-05-08T06:36:56.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.339+0000 7f56d31e4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f56cc15fb40 con 0x7f56cc15f760 2024-05-08T06:36:56.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.339+0000 7f56d31e4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f56cc0d36f0 con 0x7f56cc0d0a00 2024-05-08T06:36:56.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.339+0000 7f56d0f80640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f56cc15f760 0x7f56cc0b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:56.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.339+0000 7f56d0f80640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f56cc15f760 0x7f56cc0b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44868/0 (socket says 172.21.15.163:44868) 2024-05-08T06:36:56.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.339+0000 7f56d0f80640 1 -- 172.21.15.163:0/1740004667 learned_addr learned my addr 172.21.15.163:0/1740004667 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:56.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.340+0000 7f56d1781640 1 --2- 172.21.15.163:0/1740004667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56cc0d0a00 0x7f56cc0b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:56.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.340+0000 7f56d0f80640 1 -- 172.21.15.163:0/1740004667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56cc0d0a00 msgr2=0x7f56cc0b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:56.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.340+0000 7f56d0f80640 1 --2- 172.21.15.163:0/1740004667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56cc0d0a00 0x7f56cc0b8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.340+0000 7f56d0f80640 1 -- 172.21.15.163:0/1740004667 >> v1:172.21.15.192:6789/0 conn(0x7f56cc0cfc10 legacy=0x7f56cc0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:56.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.340+0000 7f56d0f80640 1 -- 172.21.15.163:0/1740004667 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc0d39d0 con 0x7f56cc15f760 2024-05-08T06:36:56.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.340+0000 7f56d0f80640 1 --2- 172.21.15.163:0/1740004667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56cc15f760 0x7f56cc0b4770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009870 tx=0x7f56bc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6992cebca43bec86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:56.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.340+0000 7f56cb7fe640 1 -- 172.21.15.163:0/1740004667 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc037070 con 0x7f56cc15f760 2024-05-08T06:36:56.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.340+0000 7f56cb7fe640 1 -- 172.21.15.163:0/1740004667 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56bc032440 con 0x7f56cc15f760 2024-05-08T06:36:56.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.341+0000 7f56cb7fe640 1 -- 172.21.15.163:0/1740004667 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc032810 con 0x7f56cc15f760 2024-05-08T06:36:56.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.341+0000 7f56d31e4640 1 -- 172.21.15.163:0/1740004667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56cc15f760 msgr2=0x7f56cc0b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:56.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.341+0000 7f56d31e4640 1 --2- 172.21.15.163:0/1740004667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56cc15f760 0x7f56cc0b4770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009870 tx=0x7f56bc02a1c0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.341+0000 7f56d31e4640 1 -- 172.21.15.163:0/1740004667 shutdown_connections 2024-05-08T06:36:56.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.341+0000 7f56d31e4640 1 --2- 172.21.15.163:0/1740004667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56cc0d0a00 0x7f56cc0b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.341+0000 7f56d31e4640 1 --2- 172.21.15.163:0/1740004667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56cc15f760 0x7f56cc0b4770 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.341+0000 7f56d31e4640 1 -- 172.21.15.163:0/1740004667 >> 172.21.15.163:0/1740004667 conn(0x7f56cc0740e0 msgr2=0x7f56cc06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:56.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.341+0000 7f56d31e4640 1 -- 172.21.15.163:0/1740004667 shutdown_connections 2024-05-08T06:36:56.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.341+0000 7f56d31e4640 1 -- 172.21.15.163:0/1740004667 wait complete. 2024-05-08T06:36:56.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.342+0000 7f56d31e4640 1 Processor -- start 2024-05-08T06:36:56.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.342+0000 7f56d31e4640 1 -- start start 2024-05-08T06:36:56.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56d31e4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56cc0cfc10 0x7f56cc1f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:56.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56d31e4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56cc0d0a00 0x7f56cc14a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:56.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56d31e4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56cc15f760 0x7f56cc159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:56.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56d31e4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc0d4270 con 0x7f56cc0d0a00 2024-05-08T06:36:56.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56d31e4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc0d3f90 con 0x7f56cc15f760 2024-05-08T06:36:56.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56d31e4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f56cc0d4790 con 0x7f56cc0cfc10 2024-05-08T06:36:56.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56cbfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56cc0d0a00 0x7f56cc14a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:56.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56cbfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56cc0d0a00 0x7f56cc14a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44884/0 (socket says 172.21.15.163:44884) 2024-05-08T06:36:56.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56cbfff640 1 -- 172.21.15.163:0/1048319998 learned_addr learned my addr 172.21.15.163:0/1048319998 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:56.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56d1781640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56cc15f760 0x7f56cc159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:56.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56d0f80640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56cc0cfc10 0x7f56cc1f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:56.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56cbfff640 1 -- 172.21.15.163:0/1048319998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56cc0cfc10 msgr2=0x7f56cc1f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:56.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56cbfff640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56cc0cfc10 0x7f56cc1f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56cbfff640 1 -- 172.21.15.163:0/1048319998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56cc15f760 msgr2=0x7f56cc159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:56.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56cbfff640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56cc15f760 0x7f56cc159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56cbfff640 1 -- 172.21.15.163:0/1048319998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc15a7b0 con 0x7f56cc0d0a00 2024-05-08T06:36:56.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.343+0000 7f56d0f80640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56cc0cfc10 0x7f56cc1f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:56.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.344+0000 7f56cbfff640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56cc0d0a00 0x7f56cc14a040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f56c000c3a0 tx=0x7f56c000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:56.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.344+0000 7f56c97fa640 1 -- 172.21.15.163:0/1048319998 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0017070 con 0x7f56cc0d0a00 2024-05-08T06:36:56.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.344+0000 7f56c97fa640 1 -- 172.21.15.163:0/1048319998 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c00052d0 con 0x7f56cc0d0a00 2024-05-08T06:36:56.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.344+0000 7f56c97fa640 1 -- 172.21.15.163:0/1048319998 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0003830 con 0x7f56cc0d0a00 2024-05-08T06:36:56.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.344+0000 7f56d31e4640 1 -- 172.21.15.163:0/1048319998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56cc15aa90 con 0x7f56cc0d0a00 2024-05-08T06:36:56.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.344+0000 7f56d31e4640 1 -- 172.21.15.163:0/1048319998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56cc149b20 con 0x7f56cc0d0a00 2024-05-08T06:36:56.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.345+0000 7f56d31e4640 1 -- 172.21.15.163:0/1048319998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56cc149d20 con 0x7f56cc0d0a00 2024-05-08T06:36:56.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.346+0000 7f56c97fa640 1 -- 172.21.15.163:0/1048319998 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f56c001a070 con 0x7f56cc0d0a00 2024-05-08T06:36:56.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.346+0000 7f56c97fa640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f56b8076300 0x7f56b80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:56.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.346+0000 7f56c97fa640 1 -- 172.21.15.163:0/1048319998 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f56c0096850 con 0x7f56cc0d0a00 2024-05-08T06:36:56.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.346+0000 7f56d0f80640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f56b8076300 0x7f56b80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:56.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.346+0000 7f56d0f80640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f56b8076300 0x7f56b80787c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f56bc034000 tx=0x7f56bc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:56.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.350+0000 7f56c97fa640 1 -- 172.21.15.163:0/1048319998 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f56cc149d20 con 0x7f56cc0d0a00 2024-05-08T06:36:56.568 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.569+0000 7f56d31e4640 1 -- 172.21.15.163:0/1048319998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56cc06a550 con 0x7f56cc0d0a00 2024-05-08T06:36:56.569 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.570+0000 7f56c97fa640 1 -- 172.21.15.163:0/1048319998 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f56cc0d0070 con 0x7f56cc0d0a00 2024-05-08T06:36:56.569 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:36:56.569 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2413407842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2413407842},{"type":"v1","addr":"172.21.15.163:6835","nonce":2413407842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2405203133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2405203133},{"type":"v1","addr":"172.21.15.163:6837","nonce":2405203133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3301087897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3301087897},{"type":"v1","addr":"172.21.15.192:6833","nonce":3301087897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:36:40.985978+0000","modified":"2024-05-08T06:36:43.900778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/354782184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":354782184},{"type":"v1","addr":"172.21.15.192:6835","nonce":354782184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-05-08T06:36:56.570 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 7 2024-05-08T06:36:56.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.572+0000 7f56d31e4640 1 -- 172.21.15.163:0/1048319998 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f56b8076300 msgr2=0x7f56b80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:56.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.572+0000 7f56d31e4640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f56b8076300 0x7f56b80787c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f56bc034000 tx=0x7f56bc02e000 comp rx=0 tx=0).stop 2024-05-08T06:36:56.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.572+0000 7f56d31e4640 1 -- 172.21.15.163:0/1048319998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56cc0d0a00 msgr2=0x7f56cc14a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:56.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.572+0000 7f56d31e4640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56cc0d0a00 0x7f56cc14a040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f56c000c3a0 tx=0x7f56c000c860 comp rx=0 tx=0).stop 2024-05-08T06:36:56.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.572+0000 7f56d31e4640 1 -- 172.21.15.163:0/1048319998 shutdown_connections 2024-05-08T06:36:56.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.572+0000 7f56d31e4640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f56b8076300 0x7f56b80787c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.572+0000 7f56d31e4640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56cc15f760 0x7f56cc159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.572+0000 7f56d31e4640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56cc0d0a00 0x7f56cc14a040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.572+0000 7f56d31e4640 1 --2- 172.21.15.163:0/1048319998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56cc0cfc10 0x7f56cc1f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:56.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.572+0000 7f56d31e4640 1 -- 172.21.15.163:0/1048319998 >> 172.21.15.163:0/1048319998 conn(0x7f56cc0740e0 msgr2=0x7f56cc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:56.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.572+0000 7f56d31e4640 1 -- 172.21.15.163:0/1048319998 shutdown_connections 2024-05-08T06:36:56.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:56.572+0000 7f56d31e4640 1 -- 172.21.15.163:0/1048319998 wait complete. 2024-05-08T06:36:56.985 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T06:36:40.985978+0000', 'modified': '2024-05-08T06:36:43.900778+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4300}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4300': {'gid': 4300, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.192:6835/354782184', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 354782184}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 354782184}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4300, 'qdb_cluster': [4300]} 2024-05-08T06:36:56.986 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T06:36:56.986 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T06:36:57.088 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.089+0000 7f8bffdb6640 1 Processor -- start 2024-05-08T06:36:57.088 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.089+0000 7f8bffdb6640 1 -- start start 2024-05-08T06:36:57.092 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bffdb6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bf815f450 0x7f8bf80b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.092 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bffdb6640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8bf80b0770 0x7f8bf80b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.092 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bffdb6640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bf80b74c0 con 0x7f8bf80af290 2024-05-08T06:36:57.092 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bffdb6640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8bf80af800 con 0x7f8bf80b0770 2024-05-08T06:36:57.092 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bffdb6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8bf80b62d0 con 0x7f8bf815f450 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfd351640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8bf80af290 0x7f8bf80b6db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57226/0 (socket says 172.21.15.163:57226) 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfd351640 1 -- 172.21.15.163:0/3326864939 learned_addr learned my addr 172.21.15.163:0/3326864939 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfe353640 1 --2- 172.21.15.163:0/3326864939 >> v2:172.21.15.192:3300/0 conn(0x7f8bf80b0770 0x7f8bf80b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/3326864939 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bf815f450 0x7f8bf80b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 -- 172.21.15.163:0/3326864939 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 765070216 0 0) 0x7f8bf80b74c0 con 0x7f8bf80af290 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 -- 172.21.15.163:0/3326864939 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8be4003520 con 0x7f8bf80af290 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 -- 172.21.15.163:0/3326864939 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1518799645 0 0) 0x7f8be4003520 con 0x7f8bf80af290 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 -- 172.21.15.163:0/3326864939 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bf815f450 msgr2=0x7f8bf80b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 --2- 172.21.15.163:0/3326864939 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bf815f450 0x7f8bf80b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 -- 172.21.15.163:0/3326864939 >> v2:172.21.15.192:3300/0 conn(0x7f8bf80b0770 msgr2=0x7f8bf80b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 --2- 172.21.15.163:0/3326864939 >> v2:172.21.15.192:3300/0 conn(0x7f8bf80b0770 0x7f8bf80b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 -- 172.21.15.163:0/3326864939 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf80b65b0 con 0x7f8bf80af290 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 -- 172.21.15.163:0/3326864939 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f8be80016f0 con 0x7f8bf80af290 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 -- 172.21.15.163:0/3326864939 >> v1:172.21.15.163:6789/0 conn(0x7f8bf80af290 legacy=0x7f8bf80b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:57.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 --2- 172.21.15.163:0/3326864939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8be4003a00 0x7f8be4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 --2- 172.21.15.163:0/3326864939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8be4004800 0x7f8be4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 --2- 172.21.15.163:0/3326864939 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bf815f450 0x7f8be40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 -- 172.21.15.163:0/3326864939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf80af800 con 0x7f8be4003a00 2024-05-08T06:36:57.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 -- 172.21.15.163:0/3326864939 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf80b62d0 con 0x7f8be4004800 2024-05-08T06:36:57.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.090+0000 7f8bfcb50640 1 -- 172.21.15.163:0/3326864939 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8be400b9c0 con 0x7f8bf815f450 2024-05-08T06:36:57.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.091+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/3326864939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8be4003a00 0x7f8be4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.091+0000 7f8bfe353640 1 --2- 172.21.15.163:0/3326864939 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bf815f450 0x7f8be40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.091+0000 7f8bfd351640 1 --2- 172.21.15.163:0/3326864939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8be4004800 0x7f8be4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.091+0000 7f8bfdb52640 1 -- 172.21.15.163:0/3326864939 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bf815f450 msgr2=0x7f8be40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:57.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.091+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/3326864939 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bf815f450 0x7f8be40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.091+0000 7f8bfdb52640 1 -- 172.21.15.163:0/3326864939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8be4004800 msgr2=0x7f8be4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:57.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.091+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/3326864939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8be4004800 0x7f8be4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.091+0000 7f8bfdb52640 1 -- 172.21.15.163:0/3326864939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be4003520 con 0x7f8be4003a00 2024-05-08T06:36:57.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.091+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/3326864939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8be4003a00 0x7f8be4003e20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009af0 tx=0x7f8be4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb8ff149804fe61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:57.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.092+0000 7f8bfcb50640 1 -- 172.21.15.163:0/3326864939 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4036070 con 0x7f8be4003a00 2024-05-08T06:36:57.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.092+0000 7f8bfcb50640 1 -- 172.21.15.163:0/3326864939 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf4031400 con 0x7f8be4003a00 2024-05-08T06:36:57.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.092+0000 7f8bfcb50640 1 -- 172.21.15.163:0/3326864939 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4009690 con 0x7f8be4003a00 2024-05-08T06:36:57.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.092+0000 7f8bffdb6640 1 -- 172.21.15.163:0/3326864939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8be4003a00 msgr2=0x7f8be4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:57.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.092+0000 7f8bffdb6640 1 --2- 172.21.15.163:0/3326864939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8be4003a00 0x7f8be4003e20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009af0 tx=0x7f8be4005830 comp rx=0 tx=0).stop 2024-05-08T06:36:57.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.092+0000 7f8bfe353640 1 -- 172.21.15.163:0/3326864939 reap_dead start 2024-05-08T06:36:57.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.092+0000 7f8bffdb6640 1 -- 172.21.15.163:0/3326864939 shutdown_connections 2024-05-08T06:36:57.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.092+0000 7f8bffdb6640 1 -- 172.21.15.163:0/3326864939 >> 172.21.15.163:0/3326864939 conn(0x7f8bf8073b40 msgr2=0x7f8bf806c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:57.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.092+0000 7f8bffdb6640 1 -- 172.21.15.163:0/3326864939 shutdown_connections 2024-05-08T06:36:57.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.092+0000 7f8bffdb6640 1 -- 172.21.15.163:0/3326864939 wait complete. 2024-05-08T06:36:57.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.093+0000 7f8bffdb6640 1 Processor -- start 2024-05-08T06:36:57.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.093+0000 7f8bffdb6640 1 -- start start 2024-05-08T06:36:57.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bffdb6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8be4003a00 0x7f8bf81626a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bffdb6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bf814d330 0x7f8bf81601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bffdb6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bf815d0d0 0x7f8bf815d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bffdb6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf80b6070 con 0x7f8bf815d0d0 2024-05-08T06:36:57.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bffdb6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf80b5ef0 con 0x7f8bf814d330 2024-05-08T06:36:57.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bffdb6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8bf80b03d0 con 0x7f8be4003a00 2024-05-08T06:36:57.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bfe353640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bf815d0d0 0x7f8bf815d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bfe353640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bf815d0d0 0x7f8bf815d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44890/0 (socket says 172.21.15.163:44890) 2024-05-08T06:36:57.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bfe353640 1 -- 172.21.15.163:0/139995957 learned_addr learned my addr 172.21.15.163:0/139995957 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:57.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/139995957 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8be4003a00 0x7f8bf81626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bfd351640 1 --2- 172.21.15.163:0/139995957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bf814d330 0x7f8bf81601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bfe353640 1 -- 172.21.15.163:0/139995957 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8be4003a00 msgr2=0x7f8bf81626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bfe353640 1 --2- 172.21.15.163:0/139995957 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8be4003a00 0x7f8bf81626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bfe353640 1 -- 172.21.15.163:0/139995957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bf814d330 msgr2=0x7f8bf81601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bfe353640 1 --2- 172.21.15.163:0/139995957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bf814d330 0x7f8bf81601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.094+0000 7f8bfe353640 1 -- 172.21.15.163:0/139995957 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf8161600 con 0x7f8bf815d0d0 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.095+0000 7f8bfe353640 1 --2- 172.21.15.163:0/139995957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bf815d0d0 0x7f8bf815d550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8076030 tx=0x7f8bf00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.095+0000 7f8bee7fc640 1 -- 172.21.15.163:0/139995957 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0003990 con 0x7f8bf815d0d0 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.095+0000 7f8bee7fc640 1 -- 172.21.15.163:0/139995957 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf001b070 con 0x7f8bf815d0d0 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.095+0000 7f8bffdb6640 1 -- 172.21.15.163:0/139995957 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf81618e0 con 0x7f8bf815d0d0 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.095+0000 7f8bee7fc640 1 -- 172.21.15.163:0/139995957 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0003b70 con 0x7f8bf815d0d0 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.095+0000 7f8bffdb6640 1 -- 172.21.15.163:0/139995957 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf8161ea0 con 0x7f8bf815d0d0 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.096+0000 7f8bee7fc640 1 -- 172.21.15.163:0/139995957 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8bf0005d10 con 0x7f8bf815d0d0 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.097+0000 7f8bee7fc640 1 --2- 172.21.15.163:0/139995957 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8bdc0762d0 0x7f8bdc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.097+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/139995957 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8bdc0762d0 0x7f8bdc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.097+0000 7f8bee7fc640 1 -- 172.21.15.163:0/139995957 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8bf0097ad0 con 0x7f8bf815d0d0 2024-05-08T06:36:57.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.097+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/139995957 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8bdc0762d0 0x7f8bdc078790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4033000 tx=0x7f8bf4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.098+0000 7f8c02bad740 1 Processor -- start 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.098+0000 7f8c02bad740 1 -- start start 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.098+0000 7f8c02bad740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 0x55c9c07fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.098+0000 7f8c02bad740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f9330 0x55c9c07fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.098+0000 7f8c02bad740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07cbbb0 0x55c9c07cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.098+0000 7f8c02bad740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c9c07358d0 con 0x55c9c07f9330 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.098+0000 7f8c02bad740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c9c06eec10 con 0x55c9c07f8f50 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.098+0000 7f8c02bad740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55c9c0731890 con 0x55c9c07cbbb0 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.098+0000 7f8bfd351640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f9330 0x55c9c07fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.098+0000 7f8bfd351640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f9330 0x55c9c07fbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44902/0 (socket says 172.21.15.163:44902) 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.098+0000 7f8bfd351640 1 -- 172.21.15.163:0/2444813176 learned_addr learned my addr 172.21.15.163:0/2444813176 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.099+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/2444813176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 0x55c9c07fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.099+0000 7f8bfe353640 1 --2- 172.21.15.163:0/2444813176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07cbbb0 0x55c9c07cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.099+0000 7f8bfd351640 1 -- 172.21.15.163:0/2444813176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07cbbb0 msgr2=0x55c9c07cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:57.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.099+0000 7f8bfd351640 1 --2- 172.21.15.163:0/2444813176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07cbbb0 0x55c9c07cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.099+0000 7f8bfd351640 1 -- 172.21.15.163:0/2444813176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 msgr2=0x55c9c07fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.099+0000 7f8bfd351640 1 --2- 172.21.15.163:0/2444813176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 0x55c9c07fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.099+0000 7f8bfd351640 1 -- 172.21.15.163:0/2444813176 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9c06f65d0 con 0x55c9c07f9330 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.099+0000 7f8bfd351640 1 --2- 172.21.15.163:0/2444813176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f9330 0x55c9c07fbca0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8be8005930 tx=0x7f8be8005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92b80a7e96c8767 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.099+0000 7f8bdb7fe640 1 -- 172.21.15.163:0/2444813176 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8010a20 con 0x55c9c07f9330 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.099+0000 7f8bdb7fe640 1 -- 172.21.15.163:0/2444813176 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be8019070 con 0x55c9c07f9330 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.099+0000 7f8bdb7fe640 1 -- 172.21.15.163:0/2444813176 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8010de0 con 0x55c9c07f9330 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.100+0000 7f8c02bad740 1 -- 172.21.15.163:0/2444813176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f9330 msgr2=0x55c9c07fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.100+0000 7f8c02bad740 1 --2- 172.21.15.163:0/2444813176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f9330 0x55c9c07fbca0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8be8005930 tx=0x7f8be8005df0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.100+0000 7f8c02bad740 1 -- 172.21.15.163:0/2444813176 shutdown_connections 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.100+0000 7f8c02bad740 1 --2- 172.21.15.163:0/2444813176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07cbbb0 0x55c9c07cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.100+0000 7f8c02bad740 1 --2- 172.21.15.163:0/2444813176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f9330 0x55c9c07fbca0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.100+0000 7f8c02bad740 1 --2- 172.21.15.163:0/2444813176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 0x55c9c07fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.100+0000 7f8c02bad740 1 -- 172.21.15.163:0/2444813176 >> 172.21.15.163:0/2444813176 conn(0x55c9c07cde20 msgr2=0x55c9c0816760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.100+0000 7f8c02bad740 1 -- 172.21.15.163:0/2444813176 shutdown_connections 2024-05-08T06:36:57.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.100+0000 7f8c02bad740 1 -- 172.21.15.163:0/2444813176 wait complete. 2024-05-08T06:36:57.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.100+0000 7f8c02bad740 1 -- start start 2024-05-08T06:36:57.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.100+0000 7f8c02bad740 1 Processor -- start 2024-05-08T06:36:57.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.101+0000 7f8c02bad740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 0x55c9c07f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.101+0000 7f8c02bad740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f9330 0x55c9c0813110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.101+0000 7f8c02bad740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f3b10 0x55c9c0813650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.101+0000 7f8c02bad740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c9c07d0fd0 con 0x55c9c07f3b10 2024-05-08T06:36:57.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.101+0000 7f8c02bad740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c9c06f2760 con 0x55c9c07f8f50 2024-05-08T06:36:57.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.101+0000 7f8c02bad740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55c9c07d12b0 con 0x55c9c07f9330 2024-05-08T06:36:57.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.101+0000 7f8c02bad740 10 client.0 _ll_register_callbacks cb 0x55c9c072bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:36:57.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.101+0000 7f8bfe353640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f3b10 0x55c9c0813650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.101+0000 7f8bfe353640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f3b10 0x55c9c0813650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44916/0 (socket says 172.21.15.163:44916) 2024-05-08T06:36:57.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.101+0000 7f8bfe353640 1 -- 172.21.15.163:0/577469576 learned_addr learned my addr 172.21.15.163:0/577469576 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:57.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/577469576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 0x55c9c07f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bfd351640 1 --2- 172.21.15.163:0/577469576 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f9330 0x55c9c0813110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bfe353640 1 -- 172.21.15.163:0/577469576 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f9330 msgr2=0x55c9c0813110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bfe353640 1 --2- 172.21.15.163:0/577469576 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f9330 0x55c9c0813110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bfe353640 1 -- 172.21.15.163:0/577469576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 msgr2=0x55c9c07f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bfe353640 1 --2- 172.21.15.163:0/577469576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 0x55c9c07f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bfe353640 1 -- 172.21.15.163:0/577469576 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9c04472d0 con 0x55c9c07f3b10 2024-05-08T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/577469576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 0x55c9c07f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bfe353640 1 --2- 172.21.15.163:0/577469576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f3b10 0x55c9c0813650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8bf005ce40 tx=0x7f8bf00608d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2b8751d4d7d91e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bd8ff9640 1 -- 172.21.15.163:0/577469576 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0060220 con 0x55c9c07f3b10 2024-05-08T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bd8ff9640 1 -- 172.21.15.163:0/577469576 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8bf0067370 con 0x55c9c07f3b10 2024-05-08T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8c02bad740 1 -- 172.21.15.163:0/577469576 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8be8019070 con 0x55c9c07f3b10 2024-05-08T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bd8ff9640 1 -- 172.21.15.163:0/577469576 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f8bf0081050 con 0x55c9c07f3b10 2024-05-08T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8c02bad740 10 client.4584 fetch_fsmap learned FSMap version 7 2024-05-08T06:36:57.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8c02bad740 1 -- 172.21.15.163:0/577469576 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55c9c05116b0 con 0x55c9c07f3b10 2024-05-08T06:36:57.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bd8ff9640 1 -- 172.21.15.163:0/577469576 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be8019070 con 0x55c9c07f3b10 2024-05-08T06:36:57.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8bd8ff9640 1 -- 172.21.15.163:0/577469576 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7f8bf0082240 con 0x55c9c07f3b10 2024-05-08T06:36:57.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8c02bad740 10 client.4584 fetch_fsmap finished waiting for FSMap version 7 2024-05-08T06:36:57.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8c02bad740 10 client.4584 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T06:36:57.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8c02bad740 20 client.4584 populate_metadata read hostname 'smithi163' 2024-05-08T06:36:57.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8c02bad740 1 --2- 172.21.15.163:0/577469576 >> [v2:172.21.15.192:6834/354782184,v1:172.21.15.192:6835/354782184] conn(0x55c9c0817fd0 0x55c9c0834f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8c02bad740 4 client.4584 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:36:57.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.102+0000 7f8c02bad740 1 -- 172.21.15.163:0/577469576 --> [v2:172.21.15.192:6834/354782184,v1:172.21.15.192:6835/354782184] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9c06b9760 con 0x55c9c0817fd0 2024-05-08T06:36:57.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.103+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/577469576 >> [v2:172.21.15.192:6834/354782184,v1:172.21.15.192:6835/354782184] conn(0x55c9c0817fd0 0x55c9c0834f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.103+0000 7f8bd8ff9640 10 client.4584 ms_handle_connect on v2:172.21.15.192:6834/354782184 2024-05-08T06:36:57.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.103+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/577469576 >> [v2:172.21.15.192:6834/354782184,v1:172.21.15.192:6835/354782184] conn(0x55c9c0817fd0 0x55c9c0834f80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0349560f7a6e509 server_cookie=1334581bd6382172 in_seq=0 out_seq=0 2024-05-08T06:36:57.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8bd8ff9640 1 -- 172.21.15.163:0/577469576 <== mds.0 v2:172.21.15.192:6834/354782184 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c9c06b9760 con 0x55c9c0817fd0 2024-05-08T06:36:57.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8bd8ff9640 10 client.4584 handle_command_reply: tid=0 2024-05-08T06:36:57.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8c02bad740 1 client.4584 shutdown 2024-05-08T06:36:57.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8c02bad740 1 -- 172.21.15.163:0/577469576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f3b10 msgr2=0x55c9c0813650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:57.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8c02bad740 1 --2- 172.21.15.163:0/577469576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f3b10 0x55c9c0813650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8bf005ce40 tx=0x7f8bf00608d0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8c02bad740 1 -- 172.21.15.163:0/577469576 shutdown_connections 2024-05-08T06:36:57.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8c02bad740 1 --2- 172.21.15.163:0/577469576 >> [v2:172.21.15.192:6834/354782184,v1:172.21.15.192:6835/354782184] conn(0x55c9c0817fd0 0x55c9c0834f80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8c02bad740 1 --2- 172.21.15.163:0/577469576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f3b10 0x55c9c0813650 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8bd8ff9640 0 client.4584 ms_handle_reset on v2:172.21.15.192:6834/354782184 2024-05-08T06:36:57.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8c02bad740 1 --2- 172.21.15.163:0/577469576 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f9330 0x55c9c0813110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8c02bad740 1 --2- 172.21.15.163:0/577469576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 0x55c9c07f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8c02bad740 1 -- 172.21.15.163:0/577469576 >> 172.21.15.163:0/577469576 conn(0x55c9c07cbbb0 msgr2=0x55c9c07f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:57.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8c02bad740 1 -- 172.21.15.163:0/577469576 shutdown_connections 2024-05-08T06:36:57.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8c02bad740 1 -- 172.21.15.163:0/577469576 wait complete. 2024-05-08T06:36:57.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.104+0000 7f8c02bad740 20 client.4584 trim_cache size 0 max 16384 2024-05-08T06:36:57.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.123+0000 7f8c02bad740 1 Processor -- start 2024-05-08T06:36:57.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.123+0000 7f8c02bad740 1 -- start start 2024-05-08T06:36:57.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8c02bad740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f3b10 0x55c9c07ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8c02bad740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f8f50 0x55c9c080c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8c02bad740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f9330 0x55c9c080c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8c02bad740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf82020b0 con 0x55c9c07f8f50 2024-05-08T06:36:57.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8c02bad740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf82010b0 con 0x55c9c07f9330 2024-05-08T06:36:57.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8c02bad740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8bf80c73c0 con 0x55c9c07f3b10 2024-05-08T06:36:57.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bfd351640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f8f50 0x55c9c080c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bfd351640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f8f50 0x55c9c080c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44926/0 (socket says 172.21.15.163:44926) 2024-05-08T06:36:57.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bfd351640 1 -- 172.21.15.163:0/2595105223 learned_addr learned my addr 172.21.15.163:0/2595105223 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:57.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bfe353640 1 --2- 172.21.15.163:0/2595105223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f9330 0x55c9c080c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/2595105223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f3b10 0x55c9c07ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bfd351640 1 -- 172.21.15.163:0/2595105223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f3b10 msgr2=0x55c9c07ef800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:57.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bfd351640 1 --2- 172.21.15.163:0/2595105223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f3b10 0x55c9c07ef800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bfd351640 1 -- 172.21.15.163:0/2595105223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f9330 msgr2=0x55c9c080c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:57.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bfd351640 1 --2- 172.21.15.163:0/2595105223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f9330 0x55c9c080c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bfd351640 1 -- 172.21.15.163:0/2595105223 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9c06d6530 con 0x55c9c07f8f50 2024-05-08T06:36:57.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/2595105223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f3b10 0x55c9c07ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:36:57.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bfd351640 1 --2- 172.21.15.163:0/2595105223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f8f50 0x55c9c080c0f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8be8016040 tx=0x7f8be8007070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f4a02466ac11f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:57.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bdaffd640 1 -- 172.21.15.163:0/2595105223 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8010ab0 con 0x55c9c07f8f50 2024-05-08T06:36:57.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bdaffd640 1 -- 172.21.15.163:0/2595105223 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be8019050 con 0x55c9c07f8f50 2024-05-08T06:36:57.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.124+0000 7f8bdaffd640 1 -- 172.21.15.163:0/2595105223 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be801e400 con 0x55c9c07f8f50 2024-05-08T06:36:57.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.125+0000 7f8c02bad740 1 -- 172.21.15.163:0/2595105223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f8f50 msgr2=0x55c9c080c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:57.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.125+0000 7f8c02bad740 1 --2- 172.21.15.163:0/2595105223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f8f50 0x55c9c080c0f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8be8016040 tx=0x7f8be8007070 comp rx=0 tx=0).stop 2024-05-08T06:36:57.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.125+0000 7f8c02bad740 1 -- 172.21.15.163:0/2595105223 shutdown_connections 2024-05-08T06:36:57.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.125+0000 7f8c02bad740 1 --2- 172.21.15.163:0/2595105223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f9330 0x55c9c080c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.125+0000 7f8c02bad740 1 --2- 172.21.15.163:0/2595105223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f8f50 0x55c9c080c0f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.125+0000 7f8c02bad740 1 --2- 172.21.15.163:0/2595105223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f3b10 0x55c9c07ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.125+0000 7f8c02bad740 1 -- 172.21.15.163:0/2595105223 >> 172.21.15.163:0/2595105223 conn(0x55c9c07cbbb0 msgr2=0x55c9c0819da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:57.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.125+0000 7f8c02bad740 1 -- 172.21.15.163:0/2595105223 shutdown_connections 2024-05-08T06:36:57.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.125+0000 7f8c02bad740 1 -- 172.21.15.163:0/2595105223 wait complete. 2024-05-08T06:36:57.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.126+0000 7f8c02bad740 1 -- start start 2024-05-08T06:36:57.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.126+0000 7f8c02bad740 1 Processor -- start 2024-05-08T06:36:57.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.126+0000 7f8c02bad740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f3b10 0x55c9c07cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.126+0000 7f8c02bad740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 0x55c9c07cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.126+0000 7f8c02bad740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f9330 0x55c9c07cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.126+0000 7f8c02bad740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c9c07f3030 con 0x55c9c07f9330 2024-05-08T06:36:57.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.126+0000 7f8c02bad740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf8200a60 con 0x55c9c07f8f50 2024-05-08T06:36:57.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.126+0000 7f8c02bad740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55c9c07f25c0 con 0x55c9c07f3b10 2024-05-08T06:36:57.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8c02bad740 10 client.0 _ll_register_callbacks cb 0x7f8bf005cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:36:57.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bfe353640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f9330 0x55c9c07cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bfe353640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f9330 0x55c9c07cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44938/0 (socket says 172.21.15.163:44938) 2024-05-08T06:36:57.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bfe353640 1 -- 172.21.15.163:0/1382373684 learned_addr learned my addr 172.21.15.163:0/1382373684 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/1382373684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f3b10 0x55c9c07cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bfd351640 1 --2- 172.21.15.163:0/1382373684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 0x55c9c07cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bfe353640 1 -- 172.21.15.163:0/1382373684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f3b10 msgr2=0x55c9c07cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bfe353640 1 --2- 172.21.15.163:0/1382373684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f3b10 0x55c9c07cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bfe353640 1 -- 172.21.15.163:0/1382373684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 msgr2=0x55c9c07cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bfe353640 1 --2- 172.21.15.163:0/1382373684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 0x55c9c07cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bfe353640 1 -- 172.21.15.163:0/1382373684 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9c0828a20 con 0x55c9c07f9330 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bfe353640 1 --2- 172.21.15.163:0/1382373684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f9330 0x55c9c07cf3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0060d50 tx=0x7f8bf0079000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3d785c485f419b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bd8ff9640 1 -- 172.21.15.163:0/1382373684 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf006d040 con 0x55c9c07f9330 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bd8ff9640 1 -- 172.21.15.163:0/1382373684 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8bf007bbd0 con 0x55c9c07f9330 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8c02bad740 1 -- 172.21.15.163:0/1382373684 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8be8019050 con 0x55c9c07f9330 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bd8ff9640 1 -- 172.21.15.163:0/1382373684 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f8bf0060710 con 0x55c9c07f9330 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8c02bad740 10 client.4590 fetch_fsmap learned FSMap version 7 2024-05-08T06:36:57.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8c02bad740 1 -- 172.21.15.163:0/1382373684 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55c9c080ce80 con 0x55c9c07f9330 2024-05-08T06:36:57.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bd8ff9640 1 -- 172.21.15.163:0/1382373684 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be8019050 con 0x55c9c07f9330 2024-05-08T06:36:57.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8bd8ff9640 1 -- 172.21.15.163:0/1382373684 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7f8bf0060020 con 0x55c9c07f9330 2024-05-08T06:36:57.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8c02bad740 10 client.4590 fetch_fsmap finished waiting for FSMap version 7 2024-05-08T06:36:57.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8c02bad740 10 client.4590 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T06:36:57.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8c02bad740 20 client.4590 populate_metadata read hostname 'smithi163' 2024-05-08T06:36:57.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8c02bad740 1 --2- 172.21.15.163:0/1382373684 >> [v2:172.21.15.192:6834/354782184,v1:172.21.15.192:6835/354782184] conn(0x55c9c07cff50 0x55c9c0894da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8c02bad740 4 client.4590 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T06:36:57.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.127+0000 7f8c02bad740 1 -- 172.21.15.163:0/1382373684 --> [v2:172.21.15.192:6834/354782184,v1:172.21.15.192:6835/354782184] -- command(tid 0: {"prefix": "status"}) -- 0x55c9c07434e0 con 0x55c9c07cff50 2024-05-08T06:36:57.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/1382373684 >> [v2:172.21.15.192:6834/354782184,v1:172.21.15.192:6835/354782184] conn(0x55c9c07cff50 0x55c9c0894da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8bfdb52640 1 --2- 172.21.15.163:0/1382373684 >> [v2:172.21.15.192:6834/354782184,v1:172.21.15.192:6835/354782184] conn(0x55c9c07cff50 0x55c9c0894da0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4c3f1f205758c90 server_cookie=ad7426fd9389bce3 in_seq=0 out_seq=0 2024-05-08T06:36:57.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8bd8ff9640 10 client.4590 ms_handle_connect on v2:172.21.15.192:6834/354782184 2024-05-08T06:36:57.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8bd8ff9640 1 -- 172.21.15.163:0/1382373684 <== mds.0 v2:172.21.15.192:6834/354782184 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55c9c07434e0 con 0x55c9c07cff50 2024-05-08T06:36:57.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8bd8ff9640 10 client.4590 handle_command_reply: tid=0 2024-05-08T06:36:57.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8c02bad740 1 client.4590 shutdown 2024-05-08T06:36:57.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8c02bad740 1 -- 172.21.15.163:0/1382373684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f9330 msgr2=0x55c9c07cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:57.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8c02bad740 1 --2- 172.21.15.163:0/1382373684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f9330 0x55c9c07cf3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0060d50 tx=0x7f8bf0079000 comp rx=0 tx=0).stop 2024-05-08T06:36:57.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8c02bad740 1 -- 172.21.15.163:0/1382373684 shutdown_connections 2024-05-08T06:36:57.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8c02bad740 1 --2- 172.21.15.163:0/1382373684 >> [v2:172.21.15.192:6834/354782184,v1:172.21.15.192:6835/354782184] conn(0x55c9c07cff50 0x55c9c0894da0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8c02bad740 1 --2- 172.21.15.163:0/1382373684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c9c07f9330 0x55c9c07cf3c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8c02bad740 1 --2- 172.21.15.163:0/1382373684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9c07f8f50 0x55c9c07cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8c02bad740 1 --2- 172.21.15.163:0/1382373684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c9c07f3b10 0x55c9c07cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8bd8ff9640 0 client.4590 ms_handle_reset on v2:172.21.15.192:6834/354782184 2024-05-08T06:36:57.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8c02bad740 1 -- 172.21.15.163:0/1382373684 >> 172.21.15.163:0/1382373684 conn(0x55c9c07cbbb0 msgr2=0x55c9c07ef800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:57.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.128+0000 7f8c02bad740 1 -- 172.21.15.163:0/1382373684 shutdown_connections 2024-05-08T06:36:57.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.129+0000 7f8c02bad740 1 -- 172.21.15.163:0/1382373684 wait complete. 2024-05-08T06:36:57.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.129+0000 7f8c02bad740 20 client.4590 trim_cache size 0 max 16384 2024-05-08T06:36:57.136 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:36:57.136 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "e639506e-cf71-49c6-ae4d-5af4601c36e0", 2024-05-08T06:36:57.136 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2024-05-08T06:36:57.136 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 4300, 2024-05-08T06:36:57.136 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2024-05-08T06:36:57.136 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2024-05-08T06:36:57.136 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2024-05-08T06:36:57.136 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 15.494685998, 2024-05-08T06:36:57.136 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 7, 2024-05-08T06:36:57.136 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 20, 2024-05-08T06:36:57.136 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 20, 2024-05-08T06:36:57.136 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 21.507338415 2024-05-08T06:36:57.137 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:36:57.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.129+0000 7f8bffdb6640 1 -- 172.21.15.163:0/139995957 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8bdc0762d0 msgr2=0x7f8bdc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:57.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.129+0000 7f8bffdb6640 1 --2- 172.21.15.163:0/139995957 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8bdc0762d0 0x7f8bdc078790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4033000 tx=0x7f8bf4008030 comp rx=0 tx=0).stop 2024-05-08T06:36:57.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.129+0000 7f8bffdb6640 1 -- 172.21.15.163:0/139995957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bf815d0d0 msgr2=0x7f8bf815d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:57.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.129+0000 7f8bffdb6640 1 --2- 172.21.15.163:0/139995957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bf815d0d0 0x7f8bf815d550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8076030 tx=0x7f8bf00115a0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.129+0000 7f8bffdb6640 1 -- 172.21.15.163:0/139995957 shutdown_connections 2024-05-08T06:36:57.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.129+0000 7f8bffdb6640 1 --2- 172.21.15.163:0/139995957 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8bdc0762d0 0x7f8bdc078790 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.129+0000 7f8bffdb6640 1 --2- 172.21.15.163:0/139995957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bf815d0d0 0x7f8bf815d550 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.129+0000 7f8bffdb6640 1 --2- 172.21.15.163:0/139995957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bf814d330 0x7f8bf81601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.129+0000 7f8bffdb6640 1 --2- 172.21.15.163:0/139995957 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8be4003a00 0x7f8bf81626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.129+0000 7f8bffdb6640 1 -- 172.21.15.163:0/139995957 >> 172.21.15.163:0/139995957 conn(0x7f8bf8073b40 msgr2=0x7f8bf80aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:57.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.129+0000 7f8bffdb6640 1 -- 172.21.15.163:0/139995957 shutdown_connections 2024-05-08T06:36:57.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.129+0000 7f8bffdb6640 1 -- 172.21.15.163:0/139995957 wait complete. 2024-05-08T06:36:57.140 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T06:36:57.150 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T06:36:57.150 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T06:36:57.150 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T06:36:57.150 INFO:teuthology.run_tasks:Running task kclient... 2024-05-08T06:36:57.253 INFO:tasks.kclient:Mounting kernel clients... 2024-05-08T06:36:57.253 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-08T06:36:57.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:57.254 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-05-08T06:36:57.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:57.271 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2024-05-08T06:36:57.338 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2024-05-08T06:36:57.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:36:57.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:57.340 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-05-08T06:36:57.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:57.395 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2024-05-08T06:36:57.462 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2024-05-08T06:36:57.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:36:57.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:57.464 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-05-08T06:36:57.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:57.484 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2024-05-08T06:36:57.557 INFO:teuthology.orchestra.run.smithi192.stderr:Cannot find device "ceph-brx" 2024-05-08T06:36:57.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:36:57.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:57.559 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-05-08T06:36:57.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:57.618 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2024-05-08T06:36:57.687 INFO:teuthology.orchestra.run.smithi192.stderr:Cannot find device "ceph-brx" 2024-05-08T06:36:57.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:36:57.688 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-05-08T06:36:57.689 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T06:36:57.689 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T06:36:57.689 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T06:36:57.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.790+0000 7f7d54ac4640 1 Processor -- start 2024-05-08T06:36:57.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.790+0000 7f7d54ac4640 1 -- start start 2024-05-08T06:36:57.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.791+0000 7f7d54ac4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7d50144c40 0x7f7d501450c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.791+0000 7f7d54ac4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7d50142a60 0x7f7d50142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.791+0000 7f7d54ac4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d50071e30 con 0x7f7d500d0c40 2024-05-08T06:36:57.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.791+0000 7f7d54ac4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f7d50160900 con 0x7f7d50142a60 2024-05-08T06:36:57.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.791+0000 7f7d54ac4640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f7d501459b0 con 0x7f7d50144c40 2024-05-08T06:36:57.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.791+0000 7f7d4ed9d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7d50142a60 0x7f7d50142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.791+0000 7f7d4ed9d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7d50142a60 0x7f7d50142ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44940/0 (socket says 172.21.15.163:44940) 2024-05-08T06:36:57.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.791+0000 7f7d4ed9d640 1 -- 172.21.15.163:0/1755709682 learned_addr learned my addr 172.21.15.163:0/1755709682 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:57.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.791+0000 7f7d4e59c640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f7d500d0c40 0x7f7d500d1020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57236/0 (socket says 172.21.15.163:57236) 2024-05-08T06:36:57.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.792+0000 7f7d4dd9b640 1 --2- 172.21.15.163:0/1755709682 >> v2:172.21.15.192:3300/0 conn(0x7f7d50144c40 0x7f7d501450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.792 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.792+0000 7f7d4ed9d640 1 -- 172.21.15.163:0/1755709682 >> v2:172.21.15.192:3300/0 conn(0x7f7d50144c40 msgr2=0x7f7d501450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:57.792 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.792+0000 7f7d4ed9d640 1 --2- 172.21.15.163:0/1755709682 >> v2:172.21.15.192:3300/0 conn(0x7f7d50144c40 0x7f7d501450c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.792 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.792+0000 7f7d4ed9d640 1 -- 172.21.15.163:0/1755709682 >> v1:172.21.15.163:6789/0 conn(0x7f7d500d0c40 legacy=0x7f7d500d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:57.792 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.792+0000 7f7d4d59a640 1 -- 172.21.15.163:0/1755709682 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3465102056 0 0) 0x7f7d50071e30 con 0x7f7d500d0c40 2024-05-08T06:36:57.792 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.792+0000 7f7d4ed9d640 1 -- 172.21.15.163:0/1755709682 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d50072110 con 0x7f7d50142a60 2024-05-08T06:36:57.793 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.792+0000 7f7d4ed9d640 1 --2- 172.21.15.163:0/1755709682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d50142a60 0x7f7d50142ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009870 tx=0x7f7d4802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51de7f2e8566e984 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:57.793 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.792+0000 7f7d4d59a640 1 -- 172.21.15.163:0/1755709682 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48037070 con 0x7f7d50142a60 2024-05-08T06:36:57.793 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.792+0000 7f7d4d59a640 1 -- 172.21.15.163:0/1755709682 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d48032440 con 0x7f7d50142a60 2024-05-08T06:36:57.793 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.792+0000 7f7d4d59a640 1 -- 172.21.15.163:0/1755709682 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48032810 con 0x7f7d50142a60 2024-05-08T06:36:57.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.793+0000 7f7d54ac4640 1 -- 172.21.15.163:0/1755709682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d50142a60 msgr2=0x7f7d50142ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:57.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.793+0000 7f7d54ac4640 1 --2- 172.21.15.163:0/1755709682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d50142a60 0x7f7d50142ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009870 tx=0x7f7d4802a2a0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.793+0000 7f7d54ac4640 1 -- 172.21.15.163:0/1755709682 shutdown_connections 2024-05-08T06:36:57.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.793+0000 7f7d54ac4640 1 --2- 172.21.15.163:0/1755709682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d50142a60 0x7f7d50142ef0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.793+0000 7f7d54ac4640 1 --2- 172.21.15.163:0/1755709682 >> v2:172.21.15.192:3300/0 conn(0x7f7d50144c40 0x7f7d501450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.793+0000 7f7d54ac4640 1 -- 172.21.15.163:0/1755709682 >> 172.21.15.163:0/1755709682 conn(0x7f7d500b4da0 msgr2=0x7f7d500b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:57.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.793+0000 7f7d54ac4640 1 -- 172.21.15.163:0/1755709682 shutdown_connections 2024-05-08T06:36:57.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.793+0000 7f7d54ac4640 1 -- 172.21.15.163:0/1755709682 wait complete. 2024-05-08T06:36:57.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.794+0000 7f7d54ac4640 1 Processor -- start 2024-05-08T06:36:57.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.794+0000 7f7d54ac4640 1 -- start start 2024-05-08T06:36:57.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.795+0000 7f7d54ac4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d500d0c40 0x7f7d50155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.795+0000 7f7d54ac4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d50142a60 0x7f7d501ea660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.795+0000 7f7d54ac4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d50144c40 0x7f7d50148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.795+0000 7f7d54ac4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50145600 con 0x7f7d50144c40 2024-05-08T06:36:57.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.795+0000 7f7d54ac4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50072760 con 0x7f7d500d0c40 2024-05-08T06:36:57.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.795+0000 7f7d54ac4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7d50156020 con 0x7f7d50142a60 2024-05-08T06:36:57.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.795+0000 7f7d4ed9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d50144c40 0x7f7d50148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.795+0000 7f7d4ed9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d50144c40 0x7f7d50148260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44954/0 (socket says 172.21.15.163:44954) 2024-05-08T06:36:57.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.795+0000 7f7d4ed9d640 1 -- 172.21.15.163:0/3541771244 learned_addr learned my addr 172.21.15.163:0/3541771244 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:57.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.795+0000 7f7d4e59c640 1 --2- 172.21.15.163:0/3541771244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d500d0c40 0x7f7d50155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.795+0000 7f7d4ed9d640 1 -- 172.21.15.163:0/3541771244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d50142a60 msgr2=0x7f7d501ea660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:57.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.795+0000 7f7d4ed9d640 1 --2- 172.21.15.163:0/3541771244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d50142a60 0x7f7d501ea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.795+0000 7f7d4ed9d640 1 -- 172.21.15.163:0/3541771244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d500d0c40 msgr2=0x7f7d50155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:57.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.795+0000 7f7d4ed9d640 1 --2- 172.21.15.163:0/3541771244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d500d0c40 0x7f7d50155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:57.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.796+0000 7f7d4ed9d640 1 -- 172.21.15.163:0/3541771244 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d501eaf20 con 0x7f7d50144c40 2024-05-08T06:36:57.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.796+0000 7f7d4ed9d640 1 --2- 172.21.15.163:0/3541771244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d50144c40 0x7f7d50148260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009870 tx=0x7f7d480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:57.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.796+0000 7f7d46ffd640 1 -- 172.21.15.163:0/3541771244 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48037040 con 0x7f7d50144c40 2024-05-08T06:36:57.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.796+0000 7f7d46ffd640 1 -- 172.21.15.163:0/3541771244 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d48042070 con 0x7f7d50144c40 2024-05-08T06:36:57.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.796+0000 7f7d54ac4640 1 -- 172.21.15.163:0/3541771244 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d50145d60 con 0x7f7d50144c40 2024-05-08T06:36:57.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.796+0000 7f7d46ffd640 1 -- 172.21.15.163:0/3541771244 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48030bd0 con 0x7f7d50144c40 2024-05-08T06:36:57.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.796+0000 7f7d54ac4640 1 -- 172.21.15.163:0/3541771244 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d50146320 con 0x7f7d50144c40 2024-05-08T06:36:57.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.797+0000 7f7d54ac4640 1 -- 172.21.15.163:0/3541771244 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d501426d0 con 0x7f7d50144c40 2024-05-08T06:36:57.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.797+0000 7f7d46ffd640 1 -- 172.21.15.163:0/3541771244 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7d48047460 con 0x7f7d50144c40 2024-05-08T06:36:57.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.798+0000 7f7d46ffd640 1 --2- 172.21.15.163:0/3541771244 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7d300762d0 0x7f7d30078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:57.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.798+0000 7f7d46ffd640 1 -- 172.21.15.163:0/3541771244 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7d480b8250 con 0x7f7d50144c40 2024-05-08T06:36:57.799 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.798+0000 7f7d4e59c640 1 --2- 172.21.15.163:0/3541771244 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7d300762d0 0x7f7d30078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:57.799 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.798+0000 7f7d4e59c640 1 --2- 172.21.15.163:0/3541771244 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7d300762d0 0x7f7d30078790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7d400012e0 tx=0x7f7d40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:57.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:57.802+0000 7f7d46ffd640 1 -- 172.21.15.163:0/3541771244 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7d501426d0 con 0x7f7d50144c40 2024-05-08T06:36:58.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58.020+0000 7f7d54ac4640 1 -- 172.21.15.163:0/3541771244 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7d5006a4b0 con 0x7f7d50144c40 2024-05-08T06:36:58.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58.021+0000 7f7d46ffd640 1 -- 172.21.15.163:0/3541771244 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f7d48081bb0 con 0x7f7d50144c40 2024-05-08T06:36:58.020 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T06:36:58.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58.023+0000 7f7d54ac4640 1 -- 172.21.15.163:0/3541771244 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7d300762d0 msgr2=0x7f7d30078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:58.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58.023+0000 7f7d54ac4640 1 --2- 172.21.15.163:0/3541771244 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7d300762d0 0x7f7d30078790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7d400012e0 tx=0x7f7d40009040 comp rx=0 tx=0).stop 2024-05-08T06:36:58.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58.023+0000 7f7d54ac4640 1 -- 172.21.15.163:0/3541771244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d50144c40 msgr2=0x7f7d50148260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:58.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58.023+0000 7f7d54ac4640 1 --2- 172.21.15.163:0/3541771244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d50144c40 0x7f7d50148260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009870 tx=0x7f7d480046c0 comp rx=0 tx=0).stop 2024-05-08T06:36:58.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58.023+0000 7f7d54ac4640 1 -- 172.21.15.163:0/3541771244 shutdown_connections 2024-05-08T06:36:58.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58.023+0000 7f7d54ac4640 1 --2- 172.21.15.163:0/3541771244 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7d300762d0 0x7f7d30078790 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:58.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58.023+0000 7f7d54ac4640 1 --2- 172.21.15.163:0/3541771244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d50144c40 0x7f7d50148260 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:58.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58.023+0000 7f7d54ac4640 1 --2- 172.21.15.163:0/3541771244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d50142a60 0x7f7d501ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:58.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58.023+0000 7f7d54ac4640 1 --2- 172.21.15.163:0/3541771244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d500d0c40 0x7f7d50155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:58.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58.023+0000 7f7d54ac4640 1 -- 172.21.15.163:0/3541771244 >> 172.21.15.163:0/3541771244 conn(0x7f7d500b4da0 msgr2=0x7f7d500b7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:58.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58.023+0000 7f7d54ac4640 1 -- 172.21.15.163:0/3541771244 shutdown_connections 2024-05-08T06:36:58.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58.023+0000 7f7d54ac4640 1 -- 172.21.15.163:0/3541771244 wait complete. 2024-05-08T06:36:58.033 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T06:36:58.033 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T06:36:58.033 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2024-05-08T06:36:58.033 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T06:36:58.034 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T06:36:58.034 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T06:36:58.034 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T06:36:58.034 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T06:36:58.034 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T06:36:58.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:58.034 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2024-05-08T06:36:58.050 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T06:36:58.050 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T06:36:58.050 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T06:36:58.050 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:36:58.050 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2024-05-08T06:36:58.050 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:36:58.050 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:36:58.050 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2024-05-08T06:36:58.050 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2024-05-08T06:36:58.050 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:36:58.050 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2024-05-08T06:36:58.051 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2024-05-08T06:36:58.051 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:36:58.051 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T06:36:58.051 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T06:36:58.051 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T06:36:58.051 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-08T06:36:58.051 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42768sec preferred_lft 42768sec 2024-05-08T06:36:58.051 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2024-05-08T06:36:58.051 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:36:58.051 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T06:36:58.052 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:36:58.052 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-05-08T06:36:58.052 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add name ceph-brx type bridge 2024-05-08T06:36:58.052 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip addr flush dev ceph-brx 2024-05-08T06:36:58.052 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set ceph-brx up 2024-05-08T06:36:58.052 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T06:36:58.052 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-05-08T06:36:58.139 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:36:58.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:36:58.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:58.257 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T06:36:58.281 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-05-08T06:36:58.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:58.283 DEBUG:teuthology.orchestra.run.smithi163:> route 2024-05-08T06:36:58.357 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2024-05-08T06:36:58.357 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T06:36:58.358 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T06:36:58.358 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T06:36:58.358 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T06:36:58.358 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:36:58.358 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-05-08T06:36:58.358 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T06:36:58.358 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T06:36:58.359 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T06:36:58.359 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-05-08T06:36:58.447 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:36:58.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:36:58.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:58.616 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-05-08T06:36:58.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:58.632 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list-id 2024-05-08T06:36:58.688 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:36:58.688 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-05-08T06:36:58.689 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T06:36:58.689 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T06:36:58.689 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-05-08T06:36:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:36:58.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:36:58.807 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T06:36:58.808 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:36:58.808 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-05-08T06:36:58.808 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T06:36:58.808 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T06:36:58.808 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T06:36:58.808 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T06:36:58.808 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T06:36:58.808 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-05-08T06:36:58.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:36:58.926 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:36:58.932 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:36:58.933 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-05-08T06:36:58.933 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2024-05-08T06:36:58.933 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T06:36:58.933 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-05-08T06:36:58.983 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:36:59.004 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:36:59.011 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T06:36:59.011 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T06:36:59.025 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T06:36:59.026 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T06:36:59.026 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T06:36:59.081 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T06:36:59.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:59.081 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T06:36:59.179 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T06:36:59.179 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T06:36:59.179 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-05-08T06:36:59.301 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T06:36:59.301 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T06:36:59.301 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-05-08T06:36:59.301 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.192:3300,172.21.15.192:3301" 2024-05-08T06:36:59.301 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T06:36:59.311 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T06:36:59.311 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T06:36:59.311 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-05-08T06:36:59.311 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.192:3300,172.21.15.192:3301" 2024-05-08T06:36:59.311 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T06:36:59.312 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.192:6789,172.21.15.192:6790" 2024-05-08T06:36:59.312 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.192:6789,172.21.15.192:6790:/ 2024-05-08T06:36:59.312 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e639506e-cf71-49c6-ae4d-5af4601c36e0" will pass to kernel 2024-05-08T06:36:59.312 INFO:teuthology.orchestra.run.smithi163.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T06:36:59.312 INFO:teuthology.orchestra.run.smithi163.stdout: You just mounted a file system that supports labels which does not 2024-05-08T06:36:59.313 INFO:teuthology.orchestra.run.smithi163.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T06:36:59.313 INFO:teuthology.orchestra.run.smithi163.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T06:36:59.313 INFO:teuthology.orchestra.run.smithi163.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T06:36:59.317 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T06:36:59.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:36:59.317 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T06:36:59.343 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:36:59.435 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:36:59.441 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:36:59.447 INFO:teuthology.orchestra.run.smithi163.stdout:client4585 2024-05-08T06:36:59.447 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:36:59.447 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:36:59.463 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:36:59.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:36:59.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:36:59.567 INFO:teuthology.orchestra.run.smithi163.stdout:client4585 2024-05-08T06:36:59.567 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:36:59.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:36:59.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:36:59.631 INFO:teuthology.orchestra.run.smithi163.stdout:client4585 2024-05-08T06:36:59.631 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e639506e-cf71-49c6-ae4d-5af4601c36e0.client4585/status') 2024-05-08T06:36:59.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:36:59.695 INFO:teuthology.orchestra.run.smithi163.stdout:instance: client.4585 (3)172.21.15.163:0/921220236 2024-05-08T06:36:59.695 INFO:teuthology.orchestra.run.smithi163.stdout:blocklisted: false 2024-05-08T06:36:59.696 INFO:teuthology.orchestra.run.smithi163.stderr:0+1 records in 2024-05-08T06:36:59.696 INFO:teuthology.orchestra.run.smithi163.stderr:0+1 records out 2024-05-08T06:36:59.696 INFO:teuthology.orchestra.run.smithi163.stderr:70 bytes copied, 3.9412e-05 s, 1.8 MB/s 2024-05-08T06:36:59.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:36:59.702 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-05-08T06:36:59.703 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T06:36:59.703 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-08T06:36:59.703 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T06:36:59.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.804+0000 7fd730ce8640 1 Processor -- start 2024-05-08T06:36:59.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.804+0000 7fd730ce8640 1 -- start start 2024-05-08T06:36:59.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.805+0000 7fd730ce8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd72c15f410 0x7fd72c0d4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:59.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.805+0000 7fd730ce8640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd72c0d0040 0x7fd72c0d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:59.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.805+0000 7fd730ce8640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd72c0d3b20 con 0x7fd72c0d3640 2024-05-08T06:36:59.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.805+0000 7fd730ce8640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fd72c0d3e00 con 0x7fd72c0d0040 2024-05-08T06:36:59.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.805+0000 7fd730ce8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd72c0d40e0 con 0x7fd72c15f410 2024-05-08T06:36:59.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.806+0000 7fd72ad9d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd72c0d0040 0x7fd72c0d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.806+0000 7fd729d9b640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd72c0d3640 0x7fd72c0bab90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45846/0 (socket says 172.21.15.163:45846) 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.806+0000 7fd72ad9d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd72c0d0040 0x7fd72c0d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41318/0 (socket says 172.21.15.163:41318) 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.806+0000 7fd729d9b640 1 -- 172.21.15.163:0/2692332072 learned_addr learned my addr 172.21.15.163:0/2692332072 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.806+0000 7fd72a59c640 1 --2- 172.21.15.163:0/2692332072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd72c15f410 0x7fd72c0d4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.806+0000 7fd72ad9d640 1 -- 172.21.15.163:0/2692332072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd72c15f410 msgr2=0x7fd72c0d4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.806+0000 7fd72ad9d640 1 --2- 172.21.15.163:0/2692332072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd72c15f410 0x7fd72c0d4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.806+0000 7fd72ad9d640 1 -- 172.21.15.163:0/2692332072 >> v1:172.21.15.163:6789/0 conn(0x7fd72c0d3640 legacy=0x7fd72c0bab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.806+0000 7fd72ad9d640 1 -- 172.21.15.163:0/2692332072 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c0d43c0 con 0x7fd72c0d0040 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.806+0000 7fd72959a640 1 -- 172.21.15.163:0/2692332072 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 673711123 0 0) 0x7fd72c0d3b20 con 0x7fd72c0d3640 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.807+0000 7fd72ad9d640 1 --2- 172.21.15.163:0/2692332072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd72c0d0040 0x7fd72c0d04f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0097a0 tx=0x7fd71c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca4cac69b67d827 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.807+0000 7fd72959a640 1 -- 172.21.15.163:0/2692332072 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c037070 con 0x7fd72c0d0040 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.807+0000 7fd72959a640 1 -- 172.21.15.163:0/2692332072 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd71c032440 con 0x7fd72c0d0040 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.807+0000 7fd72959a640 1 -- 172.21.15.163:0/2692332072 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c032810 con 0x7fd72c0d0040 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.807+0000 7fd730ce8640 1 -- 172.21.15.163:0/2692332072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd72c0d0040 msgr2=0x7fd72c0d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:59.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.807+0000 7fd730ce8640 1 --2- 172.21.15.163:0/2692332072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd72c0d0040 0x7fd72c0d04f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0097a0 tx=0x7fd71c02a180 comp rx=0 tx=0).stop 2024-05-08T06:36:59.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.807+0000 7fd730ce8640 1 -- 172.21.15.163:0/2692332072 shutdown_connections 2024-05-08T06:36:59.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.807+0000 7fd730ce8640 1 --2- 172.21.15.163:0/2692332072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd72c0d0040 0x7fd72c0d04f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:59.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.807+0000 7fd730ce8640 1 --2- 172.21.15.163:0/2692332072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd72c15f410 0x7fd72c0d4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:59.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.807+0000 7fd730ce8640 1 -- 172.21.15.163:0/2692332072 >> 172.21.15.163:0/2692332072 conn(0x7fd72c073e40 msgr2=0x7fd72c074250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:36:59.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.808+0000 7fd730ce8640 1 -- 172.21.15.163:0/2692332072 shutdown_connections 2024-05-08T06:36:59.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.808+0000 7fd730ce8640 1 -- 172.21.15.163:0/2692332072 wait complete. 2024-05-08T06:36:59.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.809+0000 7fd730ce8640 1 Processor -- start 2024-05-08T06:36:59.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.809+0000 7fd730ce8640 1 -- start start 2024-05-08T06:36:59.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd730ce8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd72c0d0040 0x7fd72c14d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:59.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd730ce8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd72c0d3640 0x7fd72c0abd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:59.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd730ce8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd72c15f410 0x7fd72c161d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:59.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd730ce8640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c15f970 con 0x7fd72c0d0040 2024-05-08T06:36:59.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd730ce8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c15f7f0 con 0x7fd72c0d3640 2024-05-08T06:36:59.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd730ce8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd72c0b83a0 con 0x7fd72c15f410 2024-05-08T06:36:59.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd72a59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd72c0d0040 0x7fd72c14d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:59.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd72a59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd72c0d0040 0x7fd72c14d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41328/0 (socket says 172.21.15.163:41328) 2024-05-08T06:36:59.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd72a59c640 1 -- 172.21.15.163:0/1497139945 learned_addr learned my addr 172.21.15.163:0/1497139945 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:36:59.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd72ad9d640 1 --2- 172.21.15.163:0/1497139945 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd72c15f410 0x7fd72c161d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:59.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd729d9b640 1 --2- 172.21.15.163:0/1497139945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd72c0d3640 0x7fd72c0abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:59.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd72a59c640 1 -- 172.21.15.163:0/1497139945 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd72c15f410 msgr2=0x7fd72c161d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:59.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd72a59c640 1 --2- 172.21.15.163:0/1497139945 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd72c15f410 0x7fd72c161d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:59.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd72a59c640 1 -- 172.21.15.163:0/1497139945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd72c0d3640 msgr2=0x7fd72c0abd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:36:59.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd72a59c640 1 --2- 172.21.15.163:0/1497139945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd72c0d3640 0x7fd72c0abd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:36:59.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd72a59c640 1 -- 172.21.15.163:0/1497139945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c160cd0 con 0x7fd72c0d0040 2024-05-08T06:36:59.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd72a59c640 1 --2- 172.21.15.163:0/1497139945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd72c0d0040 0x7fd72c14d600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd724007e70 tx=0x7fd72400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:59.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd722ffd640 1 -- 172.21.15.163:0/1497139945 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724014620 con 0x7fd72c0d0040 2024-05-08T06:36:59.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd722ffd640 1 -- 172.21.15.163:0/1497139945 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd724019070 con 0x7fd72c0d0040 2024-05-08T06:36:59.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd722ffd640 1 -- 172.21.15.163:0/1497139945 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724014a00 con 0x7fd72c0d0040 2024-05-08T06:36:59.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd730ce8640 1 -- 172.21.15.163:0/1497139945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72c160fb0 con 0x7fd72c0d0040 2024-05-08T06:36:59.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.810+0000 7fd730ce8640 1 -- 172.21.15.163:0/1497139945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72c161570 con 0x7fd72c0d0040 2024-05-08T06:36:59.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.812+0000 7fd730ce8640 1 -- 172.21.15.163:0/1497139945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72c06a4b0 con 0x7fd72c0d0040 2024-05-08T06:36:59.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.812+0000 7fd722ffd640 1 -- 172.21.15.163:0/1497139945 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd724014ba0 con 0x7fd72c0d0040 2024-05-08T06:36:59.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.813+0000 7fd722ffd640 1 --2- 172.21.15.163:0/1497139945 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd70c0762d0 0x7fd70c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:36:59.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.813+0000 7fd722ffd640 1 -- 172.21.15.163:0/1497139945 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd724097e20 con 0x7fd72c0d0040 2024-05-08T06:36:59.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.813+0000 7fd729d9b640 1 --2- 172.21.15.163:0/1497139945 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd70c0762d0 0x7fd70c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:36:59.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.813+0000 7fd729d9b640 1 --2- 172.21.15.163:0/1497139945 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd70c0762d0 0x7fd70c078790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd72c15ec10 tx=0x7fd714009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:36:59.816 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:36:59.817+0000 7fd722ffd640 1 -- 172.21.15.163:0/1497139945 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd724061810 con 0x7fd72c0d0040 2024-05-08T06:37:00.034 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00.035+0000 7fd730ce8640 1 -- 172.21.15.163:0/1497139945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd72c14db40 con 0x7fd72c0d0040 2024-05-08T06:37:00.035 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00.035+0000 7fd722ffd640 1 -- 172.21.15.163:0/1497139945 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fd724061110 con 0x7fd72c0d0040 2024-05-08T06:37:00.035 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T06:37:00.037 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00.038+0000 7fd730ce8640 1 -- 172.21.15.163:0/1497139945 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd70c0762d0 msgr2=0x7fd70c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:00.037 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00.038+0000 7fd730ce8640 1 --2- 172.21.15.163:0/1497139945 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd70c0762d0 0x7fd70c078790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd72c15ec10 tx=0x7fd714009040 comp rx=0 tx=0).stop 2024-05-08T06:37:00.037 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00.038+0000 7fd730ce8640 1 -- 172.21.15.163:0/1497139945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd72c0d0040 msgr2=0x7fd72c14d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:00.037 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00.038+0000 7fd730ce8640 1 --2- 172.21.15.163:0/1497139945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd72c0d0040 0x7fd72c14d600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd724007e70 tx=0x7fd72400e4b0 comp rx=0 tx=0).stop 2024-05-08T06:37:00.037 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00.038+0000 7fd730ce8640 1 -- 172.21.15.163:0/1497139945 shutdown_connections 2024-05-08T06:37:00.037 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00.038+0000 7fd730ce8640 1 --2- 172.21.15.163:0/1497139945 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd70c0762d0 0x7fd70c078790 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:00.037 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00.038+0000 7fd730ce8640 1 --2- 172.21.15.163:0/1497139945 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd72c15f410 0x7fd72c161d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:00.038 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00.038+0000 7fd730ce8640 1 --2- 172.21.15.163:0/1497139945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd72c0d3640 0x7fd72c0abd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:00.038 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00.038+0000 7fd730ce8640 1 --2- 172.21.15.163:0/1497139945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd72c0d0040 0x7fd72c14d600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:00.038 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00.038+0000 7fd730ce8640 1 -- 172.21.15.163:0/1497139945 >> 172.21.15.163:0/1497139945 conn(0x7fd72c073e40 msgr2=0x7fd72c075410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:00.038 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00.038+0000 7fd730ce8640 1 -- 172.21.15.163:0/1497139945 shutdown_connections 2024-05-08T06:37:00.038 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00.038+0000 7fd730ce8640 1 -- 172.21.15.163:0/1497139945 wait complete. 2024-05-08T06:37:00.048 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T06:37:00.048 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T06:37:00.048 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2024-05-08T06:37:00.048 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-08T06:37:00.048 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-08T06:37:00.048 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T06:37:00.048 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T06:37:00.048 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T06:37:00.048 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-08T06:37:00.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:00.049 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2024-05-08T06:37:00.065 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T06:37:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T06:37:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T06:37:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2024-05-08T06:37:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:00.065 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:37:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2024-05-08T06:37:00.065 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:37:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:00.065 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2024-05-08T06:37:00.066 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:37:00.066 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:00.066 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T06:37:00.066 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:00.066 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-08T06:37:00.066 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42766sec preferred_lft 42766sec 2024-05-08T06:37:00.066 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2024-05-08T06:37:00.066 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:00.066 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T06:37:00.066 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 5e:ea:b2:d9:8a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:00.066 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T06:37:00.066 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:00.067 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::54c1:2dff:fec0:8aa2/64 scope link tentative 2024-05-08T06:37:00.067 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:00.067 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T06:37:00.067 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 5e:ea:b2:d9:8a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T06:37:00.067 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::5cea:b2ff:fed9:8a9c/64 scope link tentative 2024-05-08T06:37:00.067 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:00.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:00.067 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T06:37:00.128 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-05-08T06:37:00.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:00.130 DEBUG:teuthology.orchestra.run.smithi163:> route 2024-05-08T06:37:00.186 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2024-05-08T06:37:00.186 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T06:37:00.187 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T06:37:00.187 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T06:37:00.187 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T06:37:00.187 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:37:00.187 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-05-08T06:37:00.187 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T06:37:00.187 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T06:37:00.187 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T06:37:00.187 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-05-08T06:37:00.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:00.308 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:00.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:00.314 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-05-08T06:37:00.329 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T06:37:00.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:00.330 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list-id 2024-05-08T06:37:00.385 INFO:teuthology.orchestra.run.smithi163.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-08T06:37:00.386 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:37:00.386 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-05-08T06:37:00.386 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T06:37:00.386 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-08T06:37:00.386 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-05-08T06:37:00.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:00.497 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:00.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:00.504 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-08T06:37:00.528 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T06:37:00.528 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T06:37:00.528 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T06:37:00.528 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:00.528 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2024-05-08T06:37:00.528 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:00.528 INFO:teuthology.orchestra.run.smithi163.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T06:37:00.528 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether b6:1d:8d:da:c3:87 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T06:37:00.529 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T06:37:00.529 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:00.529 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::b41d:8dff:feda:c387/64 scope link 2024-05-08T06:37:00.529 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:00.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:00.530 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-08T06:37:00.594 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T06:37:00.595 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T06:37:00.595 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T06:37:00.595 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:00.595 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2024-05-08T06:37:00.595 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:00.595 INFO:teuthology.orchestra.run.smithi163.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T06:37:00.595 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether b6:1d:8d:da:c3:87 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T06:37:00.595 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T06:37:00.595 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:00.595 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::b41d:8dff:feda:c387/64 scope link 2024-05-08T06:37:00.595 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:00.596 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-08T06:37:00.597 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:37:00.597 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-05-08T06:37:00.597 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-08T06:37:00.597 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-08T06:37:00.597 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-08T06:37:00.597 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-08T06:37:00.597 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-08T06:37:00.597 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-05-08T06:37:00.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:00.745 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:00.751 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:37:00.752 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-05-08T06:37:00.752 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.1 up 2024-05-08T06:37:00.752 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set dev brx.1 master ceph-brx 2024-05-08T06:37:00.752 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-05-08T06:37:00.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:00.822 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:00.829 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T06:37:00.829 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-08T06:37:00.844 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-08T06:37:00.844 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T06:37:00.844 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-08T06:37:00.903 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T06:37:00.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:00.903 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-08T06:37:00.987 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-08T06:37:00.987 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-08T06:37:00.987 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-05-08T06:37:00.995 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-08T06:37:00.996 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-08T06:37:00.996 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-05-08T06:37:00.996 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.192:3300,172.21.15.192:3301" 2024-05-08T06:37:00.996 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T06:37:01.006 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-08T06:37:01.006 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-08T06:37:01.006 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-05-08T06:37:01.006 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.192:3300,172.21.15.192:3301" 2024-05-08T06:37:01.006 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T06:37:01.006 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.192:6789,172.21.15.192:6790" 2024-05-08T06:37:01.007 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.192:6789,172.21.15.192:6790:/ 2024-05-08T06:37:01.007 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=e639506e-cf71-49c6-ae4d-5af4601c36e0" will pass to kernel 2024-05-08T06:37:01.007 INFO:teuthology.orchestra.run.smithi163.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-05-08T06:37:01.007 INFO:teuthology.orchestra.run.smithi163.stdout: You just mounted a file system that supports labels which does not 2024-05-08T06:37:01.007 INFO:teuthology.orchestra.run.smithi163.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T06:37:01.007 INFO:teuthology.orchestra.run.smithi163.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T06:37:01.007 INFO:teuthology.orchestra.run.smithi163.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T06:37:01.008 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T06:37:01.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:01.009 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-08T06:37:01.034 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:37:01.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:37:01.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:01.131 INFO:teuthology.orchestra.run.smithi163.stdout:client4611 2024-05-08T06:37:01.131 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:37:01.131 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:01.148 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:37:01.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:37:01.238 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:01.243 INFO:teuthology.orchestra.run.smithi163.stdout:client4611 2024-05-08T06:37:01.244 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:37:01.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:37:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:01.300 INFO:teuthology.orchestra.run.smithi163.stdout:client4611 2024-05-08T06:37:01.301 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e639506e-cf71-49c6-ae4d-5af4601c36e0.client4611/status') 2024-05-08T06:37:01.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:01.359 INFO:teuthology.orchestra.run.smithi163.stdout:instance: client.4611 (3)192.168.0.2:0/2206137312 2024-05-08T06:37:01.359 INFO:teuthology.orchestra.run.smithi163.stdout:blocklisted: false 2024-05-08T06:37:01.359 INFO:teuthology.orchestra.run.smithi163.stderr:0+1 records in 2024-05-08T06:37:01.359 INFO:teuthology.orchestra.run.smithi163.stderr:0+1 records out 2024-05-08T06:37:01.359 INFO:teuthology.orchestra.run.smithi163.stderr:69 bytes copied, 2.2983e-05 s, 3.0 MB/s 2024-05-08T06:37:01.360 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:01.366 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-05-08T06:37:01.366 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T06:37:01.366 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T06:37:01.367 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T06:37:01.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.467+0000 7fea907af640 1 Processor -- start 2024-05-08T06:37:01.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.468+0000 7fea907af640 1 -- start start 2024-05-08T06:37:01.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.468+0000 7fea907af640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea880d3640 0x7fea880d4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:01.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea907af640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea880d3b20 con 0x7fea8815f410 2024-05-08T06:37:01.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea907af640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea880d3e00 con 0x7fea880d0040 2024-05-08T06:37:01.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea907af640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fea880d40e0 con 0x7fea880d3640 2024-05-08T06:37:01.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8e54b640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fea8815f410 0x7fea880bab90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45850/0 (socket says 172.21.15.163:45850) 2024-05-08T06:37:01.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8e54b640 1 -- 172.21.15.163:0/2462896483 learned_addr learned my addr 172.21.15.163:0/2462896483 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:01.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8dd4a640 1 --2- 172.21.15.163:0/2462896483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea880d3640 0x7fea880d4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:01.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 -- 172.21.15.163:0/2462896483 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1231765967 0 0) 0x7fea880d3b20 con 0x7fea8815f410 2024-05-08T06:37:01.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 -- 172.21.15.163:0/2462896483 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea70003520 con 0x7fea8815f410 2024-05-08T06:37:01.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 -- 172.21.15.163:0/2462896483 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2840604556 0 0) 0x7fea70003520 con 0x7fea8815f410 2024-05-08T06:37:01.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 -- 172.21.15.163:0/2462896483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea880d3640 msgr2=0x7fea880d4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:01.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 --2- 172.21.15.163:0/2462896483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea880d3640 0x7fea880d4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:01.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 -- 172.21.15.163:0/2462896483 >> v1:172.21.15.192:6789/0 conn(0x7fea880d0040 legacy=0x7fea880b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:01.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 -- 172.21.15.163:0/2462896483 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea880d43c0 con 0x7fea8815f410 2024-05-08T06:37:01.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 -- 172.21.15.163:0/2462896483 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7fea800016f0 con 0x7fea8815f410 2024-05-08T06:37:01.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 -- 172.21.15.163:0/2462896483 >> v1:172.21.15.163:6789/0 conn(0x7fea8815f410 legacy=0x7fea880bab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:01.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 --2- 172.21.15.163:0/2462896483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea880d3640 0x7fea700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:01.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 --2- 172.21.15.163:0/2462896483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea70003f70 0x7fea70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:01.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 --2- 172.21.15.163:0/2462896483 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea70004d40 0x7fea700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:01.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 -- 172.21.15.163:0/2462896483 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fea880d40e0 con 0x7fea70004d40 2024-05-08T06:37:01.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 -- 172.21.15.163:0/2462896483 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fea7000bad0 con 0x7fea70003f70 2024-05-08T06:37:01.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8d549640 1 -- 172.21.15.163:0/2462896483 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fea7000bcd0 con 0x7fea880d3640 2024-05-08T06:37:01.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8e54b640 1 --2- 172.21.15.163:0/2462896483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea880d3640 0x7fea700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:01.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8ed4c640 1 --2- 172.21.15.163:0/2462896483 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea70004d40 0x7fea700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:01.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.469+0000 7fea8dd4a640 1 --2- 172.21.15.163:0/2462896483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea70003f70 0x7fea70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:01.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.470+0000 7fea8ed4c640 1 -- 172.21.15.163:0/2462896483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea880d3640 msgr2=0x7fea700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:01.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.470+0000 7fea8ed4c640 1 --2- 172.21.15.163:0/2462896483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea880d3640 0x7fea700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:01.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.470+0000 7fea8ed4c640 1 -- 172.21.15.163:0/2462896483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea70003f70 msgr2=0x7fea70004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:01.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.470+0000 7fea8ed4c640 1 --2- 172.21.15.163:0/2462896483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea70003f70 0x7fea70004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:01.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.470+0000 7fea8ed4c640 1 -- 172.21.15.163:0/2462896483 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea70003520 con 0x7fea70004d40 2024-05-08T06:37:01.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.470+0000 7fea8ed4c640 1 --2- 172.21.15.163:0/2462896483 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea70004d40 0x7fea700051c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea78001860 tx=0x7fea70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f96230cb5631e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:01.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.471+0000 7fea8d549640 1 -- 172.21.15.163:0/2462896483 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78037070 con 0x7fea70004d40 2024-05-08T06:37:01.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.471+0000 7fea8d549640 1 -- 172.21.15.163:0/2462896483 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea7802abd0 con 0x7fea70004d40 2024-05-08T06:37:01.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.471+0000 7fea8d549640 1 -- 172.21.15.163:0/2462896483 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea780014b0 con 0x7fea70004d40 2024-05-08T06:37:01.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.471+0000 7fea907af640 1 -- 172.21.15.163:0/2462896483 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea70004d40 msgr2=0x7fea700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:01.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.471+0000 7fea907af640 1 --2- 172.21.15.163:0/2462896483 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea70004d40 0x7fea700051c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea78001860 tx=0x7fea70005830 comp rx=0 tx=0).stop 2024-05-08T06:37:01.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.471+0000 7fea8ed4c640 1 -- 172.21.15.163:0/2462896483 reap_dead start 2024-05-08T06:37:01.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.471+0000 7fea907af640 1 -- 172.21.15.163:0/2462896483 shutdown_connections 2024-05-08T06:37:01.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.471+0000 7fea907af640 1 -- 172.21.15.163:0/2462896483 >> 172.21.15.163:0/2462896483 conn(0x7fea88073e40 msgr2=0x7fea88074250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:01.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.471+0000 7fea907af640 1 -- 172.21.15.163:0/2462896483 shutdown_connections 2024-05-08T06:37:01.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.471+0000 7fea907af640 1 -- 172.21.15.163:0/2462896483 wait complete. 2024-05-08T06:37:01.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.472+0000 7fea907af640 1 Processor -- start 2024-05-08T06:37:01.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.472+0000 7fea907af640 1 -- start start 2024-05-08T06:37:01.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.473+0000 7fea907af640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea70004d40 0x7fea8815fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:01.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.473+0000 7fea907af640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea880cb060 0x7fea88153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:01.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.473+0000 7fea907af640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea88154050 0x7fea880b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:01.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.473+0000 7fea907af640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fea8815f970 con 0x7fea880cb060 2024-05-08T06:37:01.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.473+0000 7fea907af640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fea8815f7f0 con 0x7fea88154050 2024-05-08T06:37:01.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.473+0000 7fea907af640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fea880d4ba0 con 0x7fea70004d40 2024-05-08T06:37:01.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.473+0000 7fea8dd4a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea880cb060 0x7fea88153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:01.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.473+0000 7fea8dd4a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea880cb060 0x7fea88153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41346/0 (socket says 172.21.15.163:41346) 2024-05-08T06:37:01.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.473+0000 7fea8dd4a640 1 -- 172.21.15.163:0/603425554 learned_addr learned my addr 172.21.15.163:0/603425554 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:01.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.473+0000 7fea8dd4a640 1 -- 172.21.15.163:0/603425554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea70004d40 msgr2=0x7fea8815fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:01.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.473+0000 7fea8dd4a640 1 --2- 172.21.15.163:0/603425554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea70004d40 0x7fea8815fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:01.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.473+0000 7fea8dd4a640 1 -- 172.21.15.163:0/603425554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea88154050 msgr2=0x7fea880b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:01.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.473+0000 7fea8dd4a640 1 --2- 172.21.15.163:0/603425554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea88154050 0x7fea880b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:01.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.474+0000 7fea8dd4a640 1 -- 172.21.15.163:0/603425554 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea881f97d0 con 0x7fea880cb060 2024-05-08T06:37:01.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.474+0000 7fea8dd4a640 1 --2- 172.21.15.163:0/603425554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea880cb060 0x7fea88153b10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fea84009f90 tx=0x7fea8400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:01.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.474+0000 7fea7effd640 1 -- 172.21.15.163:0/603425554 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84019070 con 0x7fea880cb060 2024-05-08T06:37:01.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.474+0000 7fea7effd640 1 -- 172.21.15.163:0/603425554 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea84003990 con 0x7fea880cb060 2024-05-08T06:37:01.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.474+0000 7fea907af640 1 -- 172.21.15.163:0/603425554 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea881f8c50 con 0x7fea880cb060 2024-05-08T06:37:01.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.474+0000 7fea7effd640 1 -- 172.21.15.163:0/603425554 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84003d40 con 0x7fea880cb060 2024-05-08T06:37:01.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.474+0000 7fea907af640 1 -- 172.21.15.163:0/603425554 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea881f9210 con 0x7fea880cb060 2024-05-08T06:37:01.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.475+0000 7fea907af640 1 -- 172.21.15.163:0/603425554 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea88154470 con 0x7fea880cb060 2024-05-08T06:37:01.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.476+0000 7fea7effd640 1 -- 172.21.15.163:0/603425554 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fea8401c070 con 0x7fea880cb060 2024-05-08T06:37:01.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.476+0000 7fea7effd640 1 --2- 172.21.15.163:0/603425554 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fea74076300 0x7fea740787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:01.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.476+0000 7fea8e54b640 1 --2- 172.21.15.163:0/603425554 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fea74076300 0x7fea740787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:01.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.476+0000 7fea7effd640 1 -- 172.21.15.163:0/603425554 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fea84098170 con 0x7fea880cb060 2024-05-08T06:37:01.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.477+0000 7fea8e54b640 1 --2- 172.21.15.163:0/603425554 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fea74076300 0x7fea740787c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea80001b00 tx=0x7fea80001a70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:01.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.481+0000 7fea7effd640 1 -- 172.21.15.163:0/603425554 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fea84010040 con 0x7fea880cb060 2024-05-08T06:37:01.697 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.698+0000 7fea907af640 1 -- 172.21.15.163:0/603425554 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fea8806a4b0 con 0x7fea880cb060 2024-05-08T06:37:01.697 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.698+0000 7fea7effd640 1 -- 172.21.15.163:0/603425554 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fea84061b60 con 0x7fea880cb060 2024-05-08T06:37:01.697 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T06:37:01.699 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.700+0000 7fea907af640 1 -- 172.21.15.163:0/603425554 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fea74076300 msgr2=0x7fea740787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:01.699 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.700+0000 7fea907af640 1 --2- 172.21.15.163:0/603425554 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fea74076300 0x7fea740787c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea80001b00 tx=0x7fea80001a70 comp rx=0 tx=0).stop 2024-05-08T06:37:01.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.700+0000 7fea907af640 1 -- 172.21.15.163:0/603425554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea880cb060 msgr2=0x7fea88153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:01.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.700+0000 7fea907af640 1 --2- 172.21.15.163:0/603425554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea880cb060 0x7fea88153b10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fea84009f90 tx=0x7fea8400f580 comp rx=0 tx=0).stop 2024-05-08T06:37:01.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.700+0000 7fea907af640 1 -- 172.21.15.163:0/603425554 shutdown_connections 2024-05-08T06:37:01.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.700+0000 7fea907af640 1 --2- 172.21.15.163:0/603425554 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fea74076300 0x7fea740787c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:01.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.700+0000 7fea907af640 1 --2- 172.21.15.163:0/603425554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea88154050 0x7fea880b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:01.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.700+0000 7fea907af640 1 --2- 172.21.15.163:0/603425554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea880cb060 0x7fea88153b10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:01.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.700+0000 7fea907af640 1 --2- 172.21.15.163:0/603425554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea70004d40 0x7fea8815fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:01.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.700+0000 7fea907af640 1 -- 172.21.15.163:0/603425554 >> 172.21.15.163:0/603425554 conn(0x7fea88073e40 msgr2=0x7fea880cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:01.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.700+0000 7fea907af640 1 -- 172.21.15.163:0/603425554 shutdown_connections 2024-05-08T06:37:01.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:01.700+0000 7fea907af640 1 -- 172.21.15.163:0/603425554 wait complete. 2024-05-08T06:37:01.710 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T06:37:01.710 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T06:37:01.710 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2024-05-08T06:37:01.710 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-08T06:37:01.710 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T06:37:01.710 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T06:37:01.710 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T06:37:01.711 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T06:37:01.711 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T06:37:01.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:01.711 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2024-05-08T06:37:01.730 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T06:37:01.731 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T06:37:01.731 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T06:37:01.731 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:01.731 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-05-08T06:37:01.731 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:01.731 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:37:01.731 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:01.731 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2024-05-08T06:37:01.731 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:37:01.731 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:01.731 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2024-05-08T06:37:01.731 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:37:01.732 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:01.732 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T06:37:01.732 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:01.732 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-08T06:37:01.732 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42730sec preferred_lft 42730sec 2024-05-08T06:37:01.732 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2024-05-08T06:37:01.732 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:01.732 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T06:37:01.732 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:37:01.732 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-08T06:37:01.732 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add name ceph-brx type bridge 2024-05-08T06:37:01.733 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr flush dev ceph-brx 2024-05-08T06:37:01.733 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set ceph-brx up 2024-05-08T06:37:01.733 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T06:37:01.733 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-08T06:37:01.825 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:01.984 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:01.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:01.990 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T06:37:02.015 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-05-08T06:37:02.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:02.017 DEBUG:teuthology.orchestra.run.smithi192:> route 2024-05-08T06:37:02.086 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2024-05-08T06:37:02.086 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T06:37:02.087 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T06:37:02.087 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T06:37:02.087 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T06:37:02.087 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:37:02.087 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-08T06:37:02.087 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T06:37:02.087 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T06:37:02.087 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T06:37:02.087 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-08T06:37:02.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:02.366 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:02.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:02.372 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-05-08T06:37:02.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:02.388 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list-id 2024-05-08T06:37:02.444 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:37:02.445 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-08T06:37:02.445 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T06:37:02.445 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-08T06:37:02.445 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-08T06:37:02.536 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:02.560 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:02.566 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-08T06:37:02.566 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:37:02.566 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-08T06:37:02.566 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-08T06:37:02.567 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T06:37:02.567 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-08T06:37:02.567 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-08T06:37:02.567 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-08T06:37:02.567 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-08T06:37:02.618 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:02.688 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:02.694 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:37:02.694 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-08T06:37:02.694 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2024-05-08T06:37:02.694 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T06:37:02.694 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-08T06:37:02.743 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:02.766 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:02.774 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T06:37:02.774 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-08T06:37:02.789 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-08T06:37:02.789 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T06:37:02.790 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-08T06:37:02.846 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T06:37:02.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:02.846 DEBUG:teuthology.orchestra.run.smithi192:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T06:37:02.937 INFO:teuthology.orchestra.run.smithi192.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T06:37:02.937 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T06:37:02.937 INFO:teuthology.orchestra.run.smithi192.stdout:invalid new device string format 2024-05-08T06:37:03.043 INFO:teuthology.orchestra.run.smithi192.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T06:37:03.043 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T06:37:03.043 INFO:teuthology.orchestra.run.smithi192.stdout:invalid new device string format 2024-05-08T06:37:03.043 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.192:3300,172.21.15.192:3301" 2024-05-08T06:37:03.043 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T06:37:03.048 INFO:teuthology.orchestra.run.smithi192.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-08T06:37:03.048 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-08T06:37:03.048 INFO:teuthology.orchestra.run.smithi192.stdout:invalid new device string format 2024-05-08T06:37:03.048 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.192:3300,172.21.15.192:3301" 2024-05-08T06:37:03.048 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T06:37:03.048 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.192:6789,172.21.15.192:6790" 2024-05-08T06:37:03.048 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.192:6789,172.21.15.192:6790:/ 2024-05-08T06:37:03.048 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e639506e-cf71-49c6-ae4d-5af4601c36e0" will pass to kernel 2024-05-08T06:37:03.049 INFO:teuthology.orchestra.run.smithi192.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-08T06:37:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: You just mounted a file system that supports labels which does not 2024-05-08T06:37:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T06:37:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T06:37:03.049 INFO:teuthology.orchestra.run.smithi192.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T06:37:03.051 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T06:37:03.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:03.051 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-08T06:37:03.075 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:37:03.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:37:03.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:03.171 INFO:teuthology.orchestra.run.smithi192.stdout:client4620 2024-05-08T06:37:03.171 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:37:03.171 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:03.187 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:37:03.276 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:37:03.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:03.284 INFO:teuthology.orchestra.run.smithi192.stdout:client4620 2024-05-08T06:37:03.284 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:37:03.332 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:37:03.334 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:03.340 INFO:teuthology.orchestra.run.smithi192.stdout:client4620 2024-05-08T06:37:03.341 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e639506e-cf71-49c6-ae4d-5af4601c36e0.client4620/status') 2024-05-08T06:37:03.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:03.399 INFO:teuthology.orchestra.run.smithi192.stdout:instance: client.4620 (3)172.21.15.192:0/861896061 2024-05-08T06:37:03.400 INFO:teuthology.orchestra.run.smithi192.stdout:blocklisted: false 2024-05-08T06:37:03.400 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records in 2024-05-08T06:37:03.400 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records out 2024-05-08T06:37:03.400 INFO:teuthology.orchestra.run.smithi192.stderr:70 bytes copied, 3.7731e-05 s, 1.9 MB/s 2024-05-08T06:37:03.400 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:03.407 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-05-08T06:37:03.407 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T06:37:03.407 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-08T06:37:03.407 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T06:37:03.520 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.521+0000 7ffa132c8640 1 Processor -- start 2024-05-08T06:37:03.520 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.521+0000 7ffa132c8640 1 -- start start 2024-05-08T06:37:03.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa132c8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa0c159e10 0x7ffa0c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:03.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa132c8640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa0c0b5070 con 0x7ffa0c159a30 2024-05-08T06:37:03.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa132c8640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa0c15ffc0 con 0x7ffa0c06d870 2024-05-08T06:37:03.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa132c8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0c1602a0 con 0x7ffa0c159e10 2024-05-08T06:37:03.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa11064640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ffa0c159a30 0x7ffa0c0b4960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45856/0 (socket says 172.21.15.163:45856) 2024-05-08T06:37:03.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa11064640 1 -- 172.21.15.163:0/3002355022 learned_addr learned my addr 172.21.15.163:0/3002355022 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:03.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa10863640 1 --2- 172.21.15.163:0/3002355022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa0c159e10 0x7ffa0c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:03.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 -- 172.21.15.163:0/3002355022 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3432193998 0 0) 0x7ffa0c0b5070 con 0x7ffa0c159a30 2024-05-08T06:37:03.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 -- 172.21.15.163:0/3002355022 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9f8003520 con 0x7ffa0c159a30 2024-05-08T06:37:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 -- 172.21.15.163:0/3002355022 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4092895274 0 0) 0x7ff9f8003520 con 0x7ffa0c159a30 2024-05-08T06:37:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 -- 172.21.15.163:0/3002355022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa0c159e10 msgr2=0x7ffa0c06d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 --2- 172.21.15.163:0/3002355022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa0c159e10 0x7ffa0c06d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 -- 172.21.15.163:0/3002355022 >> v1:172.21.15.192:6789/0 conn(0x7ffa0c06d870 legacy=0x7ffa0c0aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 -- 172.21.15.163:0/3002355022 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c160580 con 0x7ffa0c159a30 2024-05-08T06:37:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 -- 172.21.15.163:0/3002355022 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7ffa080016f0 con 0x7ffa0c159a30 2024-05-08T06:37:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 -- 172.21.15.163:0/3002355022 >> v1:172.21.15.163:6789/0 conn(0x7ffa0c159a30 legacy=0x7ffa0c0b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 --2- 172.21.15.163:0/3002355022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9f8003a30 0x7ff9f8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 --2- 172.21.15.163:0/3002355022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa0c159e10 0x7ff9f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 --2- 172.21.15.163:0/3002355022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9f8004d90 0x7ff9f80051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 -- 172.21.15.163:0/3002355022 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c1602a0 con 0x7ff9f8003a30 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 -- 172.21.15.163:0/3002355022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f800bc50 con 0x7ff9f8004d90 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.522+0000 7ffa03fff640 1 -- 172.21.15.163:0/3002355022 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f800be50 con 0x7ffa0c159e10 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.523+0000 7ffa11064640 1 --2- 172.21.15.163:0/3002355022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9f8003a30 0x7ff9f8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.523+0000 7ffa10863640 1 --2- 172.21.15.163:0/3002355022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa0c159e10 0x7ff9f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.523+0000 7ffa11865640 1 --2- 172.21.15.163:0/3002355022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9f8004d90 0x7ff9f80051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.523+0000 7ffa11064640 1 -- 172.21.15.163:0/3002355022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa0c159e10 msgr2=0x7ff9f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.523+0000 7ffa11064640 1 --2- 172.21.15.163:0/3002355022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa0c159e10 0x7ff9f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.523+0000 7ffa11064640 1 -- 172.21.15.163:0/3002355022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9f8004d90 msgr2=0x7ff9f80051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.523+0000 7ffa11064640 1 --2- 172.21.15.163:0/3002355022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9f8004d90 0x7ff9f80051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.523+0000 7ffa11064640 1 -- 172.21.15.163:0/3002355022 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f8003520 con 0x7ff9f8003a30 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.524+0000 7ffa11064640 1 --2- 172.21.15.163:0/3002355022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9f8003a30 0x7ff9f8003e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffa08001050 tx=0x7ff9f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42022fd8331ea6be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.524+0000 7ffa03fff640 1 -- 172.21.15.163:0/3002355022 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08007d10 con 0x7ff9f8003a30 2024-05-08T06:37:03.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.524+0000 7ffa03fff640 1 -- 172.21.15.163:0/3002355022 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa08005ad0 con 0x7ff9f8003a30 2024-05-08T06:37:03.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.524+0000 7ffa03fff640 1 -- 172.21.15.163:0/3002355022 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08007070 con 0x7ff9f8003a30 2024-05-08T06:37:03.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.524+0000 7ffa132c8640 1 -- 172.21.15.163:0/3002355022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9f8003a30 msgr2=0x7ff9f8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:03.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.524+0000 7ffa132c8640 1 --2- 172.21.15.163:0/3002355022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9f8003a30 0x7ff9f8003e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffa08001050 tx=0x7ff9f8005830 comp rx=0 tx=0).stop 2024-05-08T06:37:03.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.524+0000 7ffa11865640 1 -- 172.21.15.163:0/3002355022 reap_dead start 2024-05-08T06:37:03.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.524+0000 7ffa132c8640 1 -- 172.21.15.163:0/3002355022 shutdown_connections 2024-05-08T06:37:03.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.524+0000 7ffa132c8640 1 -- 172.21.15.163:0/3002355022 >> 172.21.15.163:0/3002355022 conn(0x7ffa0c072bd0 msgr2=0x7ffa0c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:03.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.524+0000 7ffa132c8640 1 -- 172.21.15.163:0/3002355022 shutdown_connections 2024-05-08T06:37:03.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.524+0000 7ffa132c8640 1 -- 172.21.15.163:0/3002355022 wait complete. 2024-05-08T06:37:03.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.526+0000 7ffa132c8640 1 Processor -- start 2024-05-08T06:37:03.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.526+0000 7ffa132c8640 1 -- start start 2024-05-08T06:37:03.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.526+0000 7ffa132c8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9f8003a30 0x7ffa0c14e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:03.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.526+0000 7ffa132c8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa0c0b08a0 0x7ffa0c154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:03.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.526+0000 7ffa132c8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa0c152120 0x7ffa0c156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:03.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.526+0000 7ffa132c8640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c15fd60 con 0x7ffa0c0b08a0 2024-05-08T06:37:03.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.526+0000 7ffa132c8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c15fbe0 con 0x7ffa0c152120 2024-05-08T06:37:03.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.526+0000 7ffa132c8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0c06d330 con 0x7ff9f8003a30 2024-05-08T06:37:03.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.526+0000 7ffa10863640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa0c0b08a0 0x7ffa0c154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:03.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.526+0000 7ffa10863640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa0c0b08a0 0x7ffa0c154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41376/0 (socket says 172.21.15.163:41376) 2024-05-08T06:37:03.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.526+0000 7ffa10863640 1 -- 172.21.15.163:0/1020849540 learned_addr learned my addr 172.21.15.163:0/1020849540 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:03.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.527+0000 7ffa11865640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa0c152120 0x7ffa0c156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:03.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.527+0000 7ffa11064640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9f8003a30 0x7ffa0c14e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:03.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.527+0000 7ffa10863640 1 -- 172.21.15.163:0/1020849540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9f8003a30 msgr2=0x7ffa0c14e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:03.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.527+0000 7ffa10863640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9f8003a30 0x7ffa0c14e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:03.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.527+0000 7ffa10863640 1 -- 172.21.15.163:0/1020849540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa0c152120 msgr2=0x7ffa0c156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:03.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.527+0000 7ffa10863640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa0c152120 0x7ffa0c156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:03.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.527+0000 7ffa10863640 1 -- 172.21.15.163:0/1020849540 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c14e730 con 0x7ffa0c0b08a0 2024-05-08T06:37:03.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.527+0000 7ffa11865640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa0c152120 0x7ffa0c156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:03.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.527+0000 7ffa10863640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa0c0b08a0 0x7ffa0c154ec0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0119d0 tx=0x7ff9fc011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:03.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.527+0000 7ffa01ffb640 1 -- 172.21.15.163:0/1020849540 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00faf0 con 0x7ffa0c0b08a0 2024-05-08T06:37:03.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.527+0000 7ffa01ffb640 1 -- 172.21.15.163:0/1020849540 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9fc00fc90 con 0x7ffa0c0b08a0 2024-05-08T06:37:03.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.527+0000 7ffa132c8640 1 -- 172.21.15.163:0/1020849540 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0c153e20 con 0x7ffa0c0b08a0 2024-05-08T06:37:03.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.527+0000 7ffa01ffb640 1 -- 172.21.15.163:0/1020849540 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc014440 con 0x7ffa0c0b08a0 2024-05-08T06:37:03.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.527+0000 7ffa132c8640 1 -- 172.21.15.163:0/1020849540 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa0c1543e0 con 0x7ffa0c0b08a0 2024-05-08T06:37:03.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.528+0000 7ffa132c8640 1 -- 172.21.15.163:0/1020849540 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa0c06a4b0 con 0x7ffa0c0b08a0 2024-05-08T06:37:03.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.529+0000 7ffa01ffb640 1 -- 172.21.15.163:0/1020849540 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff9fc012040 con 0x7ffa0c0b08a0 2024-05-08T06:37:03.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.529+0000 7ffa01ffb640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff9f00762d0 0x7ff9f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:03.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.529+0000 7ffa11064640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff9f00762d0 0x7ff9f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:03.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.529+0000 7ffa01ffb640 1 -- 172.21.15.163:0/1020849540 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff9fc097b30 con 0x7ffa0c0b08a0 2024-05-08T06:37:03.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.530+0000 7ffa11064640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff9f00762d0 0x7ff9f0078790 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa08032000 tx=0x7ffa08009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:03.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.534+0000 7ffa01ffb640 1 -- 172.21.15.163:0/1020849540 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff9fc019030 con 0x7ffa0c0b08a0 2024-05-08T06:37:03.748 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.748+0000 7ffa132c8640 1 -- 172.21.15.163:0/1020849540 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffa0c0d0820 con 0x7ffa0c0b08a0 2024-05-08T06:37:03.748 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.749+0000 7ffa01ffb640 1 -- 172.21.15.163:0/1020849540 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7ff9fc061490 con 0x7ffa0c0b08a0 2024-05-08T06:37:03.748 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T06:37:03.750 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.751+0000 7ffa132c8640 1 -- 172.21.15.163:0/1020849540 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff9f00762d0 msgr2=0x7ff9f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:03.750 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.751+0000 7ffa132c8640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff9f00762d0 0x7ff9f0078790 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa08032000 tx=0x7ffa08009030 comp rx=0 tx=0).stop 2024-05-08T06:37:03.751 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.751+0000 7ffa132c8640 1 -- 172.21.15.163:0/1020849540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa0c0b08a0 msgr2=0x7ffa0c154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:03.751 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.751+0000 7ffa132c8640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa0c0b08a0 0x7ffa0c154ec0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0119d0 tx=0x7ff9fc011e90 comp rx=0 tx=0).stop 2024-05-08T06:37:03.751 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.751+0000 7ffa132c8640 1 -- 172.21.15.163:0/1020849540 shutdown_connections 2024-05-08T06:37:03.751 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.751+0000 7ffa132c8640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff9f00762d0 0x7ff9f0078790 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:03.751 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.751+0000 7ffa132c8640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa0c152120 0x7ffa0c156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:03.751 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.751+0000 7ffa132c8640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa0c0b08a0 0x7ffa0c154ec0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:03.751 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.751+0000 7ffa132c8640 1 --2- 172.21.15.163:0/1020849540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9f8003a30 0x7ffa0c14e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:03.751 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.751+0000 7ffa132c8640 1 -- 172.21.15.163:0/1020849540 >> 172.21.15.163:0/1020849540 conn(0x7ffa0c072bd0 msgr2=0x7ffa0c0d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:03.751 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.751+0000 7ffa132c8640 1 -- 172.21.15.163:0/1020849540 shutdown_connections 2024-05-08T06:37:03.751 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:03.751+0000 7ffa132c8640 1 -- 172.21.15.163:0/1020849540 wait complete. 2024-05-08T06:37:03.761 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T06:37:03.761 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T06:37:03.761 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2024-05-08T06:37:03.761 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-08T06:37:03.761 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-08T06:37:03.761 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T06:37:03.762 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T06:37:03.762 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T06:37:03.762 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-08T06:37:03.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:03.762 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2024-05-08T06:37:03.779 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T06:37:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T06:37:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T06:37:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:03.779 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42728sec preferred_lft 42728sec 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2024-05-08T06:37:03.780 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:03.781 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T06:37:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether aa:e3:7f:c0:cd:4b brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T06:37:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::d4cb:aaff:fe2f:d3e2/64 scope link 2024-05-08T06:37:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:03.781 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T06:37:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether aa:e3:7f:c0:cd:4b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T06:37:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::a8e3:7fff:fec0:cd4b/64 scope link tentative 2024-05-08T06:37:03.781 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:03.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:03.781 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T06:37:03.843 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-05-08T06:37:03.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:03.845 DEBUG:teuthology.orchestra.run.smithi192:> route 2024-05-08T06:37:03.902 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2024-05-08T06:37:03.902 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T06:37:03.902 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T06:37:03.902 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T06:37:03.902 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T06:37:03.902 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:37:03.902 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-08T06:37:03.903 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T06:37:03.903 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T06:37:03.903 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T06:37:03.903 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-08T06:37:03.991 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:04.023 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:04.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:04.030 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-05-08T06:37:04.046 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T06:37:04.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:04.047 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list-id 2024-05-08T06:37:04.102 INFO:teuthology.orchestra.run.smithi192.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-08T06:37:04.102 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:37:04.102 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-08T06:37:04.102 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T06:37:04.102 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-08T06:37:04.102 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-08T06:37:04.190 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:04.213 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:04.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:04.219 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-08T06:37:04.243 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T06:37:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T06:37:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T06:37:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-05-08T06:37:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:04.244 INFO:teuthology.orchestra.run.smithi192.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T06:37:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 76:58:ff:1f:69:72 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T06:37:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T06:37:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::7458:ffff:fe1f:6972/64 scope link tentative 2024-05-08T06:37:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:04.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:04.246 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-08T06:37:04.311 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T06:37:04.311 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T06:37:04.311 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T06:37:04.311 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:04.311 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-05-08T06:37:04.311 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:04.311 INFO:teuthology.orchestra.run.smithi192.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T06:37:04.311 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 76:58:ff:1f:69:72 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T06:37:04.312 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T06:37:04.312 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:04.312 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::7458:ffff:fe1f:6972/64 scope link tentative 2024-05-08T06:37:04.312 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:04.313 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-08T06:37:04.313 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:37:04.313 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-08T06:37:04.313 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-08T06:37:04.313 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-08T06:37:04.313 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-08T06:37:04.313 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-08T06:37:04.313 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-08T06:37:04.313 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-08T06:37:04.401 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:04.459 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:04.465 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:37:04.465 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-08T06:37:04.465 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.1 up 2024-05-08T06:37:04.465 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set dev brx.1 master ceph-brx 2024-05-08T06:37:04.465 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-08T06:37:04.513 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:04.536 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:04.542 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T06:37:04.542 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-08T06:37:04.558 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-08T06:37:04.558 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T06:37:04.559 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-08T06:37:04.613 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T06:37:04.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:04.614 DEBUG:teuthology.orchestra.run.smithi192:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-08T06:37:04.697 INFO:teuthology.orchestra.run.smithi192.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-08T06:37:04.697 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-08T06:37:04.697 INFO:teuthology.orchestra.run.smithi192.stdout:invalid new device string format 2024-05-08T06:37:04.705 INFO:teuthology.orchestra.run.smithi192.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-08T06:37:04.706 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-08T06:37:04.706 INFO:teuthology.orchestra.run.smithi192.stdout:invalid new device string format 2024-05-08T06:37:04.706 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.192:3300,172.21.15.192:3301" 2024-05-08T06:37:04.706 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T06:37:04.711 INFO:teuthology.orchestra.run.smithi192.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-08T06:37:04.711 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-08T06:37:04.711 INFO:teuthology.orchestra.run.smithi192.stdout:invalid new device string format 2024-05-08T06:37:04.711 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.192:3300,172.21.15.192:3301" 2024-05-08T06:37:04.711 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T06:37:04.711 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.192:6789,172.21.15.192:6790" 2024-05-08T06:37:04.711 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.192:6789,172.21.15.192:6790:/ 2024-05-08T06:37:04.711 INFO:teuthology.orchestra.run.smithi192.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=e639506e-cf71-49c6-ae4d-5af4601c36e0" will pass to kernel 2024-05-08T06:37:04.711 INFO:teuthology.orchestra.run.smithi192.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-05-08T06:37:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: You just mounted a file system that supports labels which does not 2024-05-08T06:37:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T06:37:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T06:37:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T06:37:04.714 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T06:37:04.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:04.714 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-08T06:37:04.780 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:37:04.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:37:04.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:04.877 INFO:teuthology.orchestra.run.smithi192.stdout:client4472 2024-05-08T06:37:04.877 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:37:04.877 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:04.893 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:37:04.982 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:37:04.984 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:04.990 INFO:teuthology.orchestra.run.smithi192.stdout:client4472 2024-05-08T06:37:04.990 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:37:05.039 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:37:05.040 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:05.046 INFO:teuthology.orchestra.run.smithi192.stdout:client4472 2024-05-08T06:37:05.047 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e639506e-cf71-49c6-ae4d-5af4601c36e0.client4472/status') 2024-05-08T06:37:05.095 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:05.105 INFO:teuthology.orchestra.run.smithi192.stdout:instance: client.4472 (3)192.168.0.2:0/194343187 2024-05-08T06:37:05.105 INFO:teuthology.orchestra.run.smithi192.stdout:blocklisted: false 2024-05-08T06:37:05.105 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records in 2024-05-08T06:37:05.105 INFO:teuthology.orchestra.run.smithi192.stderr:0+1 records out 2024-05-08T06:37:05.105 INFO:teuthology.orchestra.run.smithi192.stderr:68 bytes copied, 2.212e-05 s, 3.1 MB/s 2024-05-08T06:37:05.106 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:37:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:05.112 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-08T06:37:05.362 INFO:tasks.cephfs_test_runner:Starting test: test_flush (tasks.cephfs.test_flush.TestFlush) 2024-05-08T06:37:05.362 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:37:05.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.475+0000 7fcc7a738640 1 Processor -- start 2024-05-08T06:37:05.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.475+0000 7fcc7a738640 1 -- start start 2024-05-08T06:37:05.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.476+0000 7fcc7a738640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcc74162900 0x7fcc74073f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:05.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.476+0000 7fcc7a738640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc740aef80 0x7fcc740b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:05.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.476+0000 7fcc7a738640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc740b74f0 con 0x7fcc740d2170 2024-05-08T06:37:05.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.476+0000 7fcc7a738640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fcc74162ce0 con 0x7fcc74162900 2024-05-08T06:37:05.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.476+0000 7fcc7a738640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcc740b50b0 con 0x7fcc740aef80 2024-05-08T06:37:05.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.476+0000 7fcc73fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcc74162900 0x7fcc74073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:05.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.476+0000 7fcc78cd5640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fcc740d2170 0x7fcc7406c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45858/0 (socket says 172.21.15.163:45858) 2024-05-08T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.476+0000 7fcc78cd5640 1 -- 172.21.15.163:0/2237157871 learned_addr learned my addr 172.21.15.163:0/2237157871 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.476+0000 7fcc737fe640 1 --2- 172.21.15.163:0/2237157871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc740aef80 0x7fcc740b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.476+0000 7fcc73fff640 1 -- 172.21.15.163:0/2237157871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc740aef80 msgr2=0x7fcc740b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.476+0000 7fcc73fff640 1 --2- 172.21.15.163:0/2237157871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc740aef80 0x7fcc740b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.476+0000 7fcc73fff640 1 -- 172.21.15.163:0/2237157871 >> v1:172.21.15.163:6789/0 conn(0x7fcc740d2170 legacy=0x7fcc7406c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.476+0000 7fcc72ffd640 1 -- 172.21.15.163:0/2237157871 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 98969770 0 0) 0x7fcc740b74f0 con 0x7fcc740d2170 2024-05-08T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.476+0000 7fcc73fff640 1 -- 172.21.15.163:0/2237157871 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc740b5390 con 0x7fcc74162900 2024-05-08T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.477+0000 7fcc73fff640 1 --2- 172.21.15.163:0/2237157871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc74162900 0x7fcc74073f70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcc640097a0 tx=0x7fcc6402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97de2c349710698 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.477+0000 7fcc72ffd640 1 -- 172.21.15.163:0/2237157871 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64037070 con 0x7fcc74162900 2024-05-08T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.477+0000 7fcc72ffd640 1 -- 172.21.15.163:0/2237157871 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6402ae50 con 0x7fcc74162900 2024-05-08T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.477+0000 7fcc72ffd640 1 -- 172.21.15.163:0/2237157871 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64032660 con 0x7fcc74162900 2024-05-08T06:37:05.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.478+0000 7fcc7a738640 1 -- 172.21.15.163:0/2237157871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc74162900 msgr2=0x7fcc74073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:05.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.478+0000 7fcc7a738640 1 --2- 172.21.15.163:0/2237157871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc74162900 0x7fcc74073f70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcc640097a0 tx=0x7fcc6402a100 comp rx=0 tx=0).stop 2024-05-08T06:37:05.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.478+0000 7fcc7a738640 1 -- 172.21.15.163:0/2237157871 shutdown_connections 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.478+0000 7fcc7a738640 1 --2- 172.21.15.163:0/2237157871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc740aef80 0x7fcc740b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.478+0000 7fcc7a738640 1 --2- 172.21.15.163:0/2237157871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc74162900 0x7fcc74073f70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.478+0000 7fcc7a738640 1 -- 172.21.15.163:0/2237157871 >> 172.21.15.163:0/2237157871 conn(0x7fcc7406d610 msgr2=0x7fcc7406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.478+0000 7fcc7a738640 1 -- 172.21.15.163:0/2237157871 shutdown_connections 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.478+0000 7fcc7a738640 1 -- 172.21.15.163:0/2237157871 wait complete. 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.479+0000 7fcc7a738640 1 Processor -- start 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.479+0000 7fcc7a738640 1 -- start start 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc7a738640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc740aef80 0x7fcc74143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc7a738640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc740d2170 0x7fcc741f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc7a738640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc74162900 0x7fcc74145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc7a738640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcc740b71e0 con 0x7fcc740d2170 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc7a738640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7406f4d0 con 0x7fcc740aef80 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc7a738640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcc740d2600 con 0x7fcc74162900 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc737fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc740d2170 0x7fcc741f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:05.481 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc737fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc740d2170 0x7fcc741f8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41388/0 (socket says 172.21.15.163:41388) 2024-05-08T06:37:05.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc737fe640 1 -- 172.21.15.163:0/873808824 learned_addr learned my addr 172.21.15.163:0/873808824 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:05.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc78cd5640 1 --2- 172.21.15.163:0/873808824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc74162900 0x7fcc74145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:05.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc73fff640 1 --2- 172.21.15.163:0/873808824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc740aef80 0x7fcc74143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:05.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc737fe640 1 -- 172.21.15.163:0/873808824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc74162900 msgr2=0x7fcc74145850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:05.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc737fe640 1 --2- 172.21.15.163:0/873808824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc74162900 0x7fcc74145850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc737fe640 1 -- 172.21.15.163:0/873808824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc740aef80 msgr2=0x7fcc74143980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:05.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc737fe640 1 --2- 172.21.15.163:0/873808824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc740aef80 0x7fcc74143980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.480+0000 7fcc737fe640 1 -- 172.21.15.163:0/873808824 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc741f9130 con 0x7fcc740d2170 2024-05-08T06:37:05.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.481+0000 7fcc737fe640 1 --2- 172.21.15.163:0/873808824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc740d2170 0x7fcc741f8bf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800e9b0 tx=0x7fcc6800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:05.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.481+0000 7fcc70ff9640 1 -- 172.21.15.163:0/873808824 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68017070 con 0x7fcc740d2170 2024-05-08T06:37:05.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.481+0000 7fcc70ff9640 1 -- 172.21.15.163:0/873808824 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6800c9a0 con 0x7fcc740d2170 2024-05-08T06:37:05.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.481+0000 7fcc7a738640 1 -- 172.21.15.163:0/873808824 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc74145d90 con 0x7fcc740d2170 2024-05-08T06:37:05.483 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.481+0000 7fcc70ff9640 1 -- 172.21.15.163:0/873808824 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6800cd70 con 0x7fcc740d2170 2024-05-08T06:37:05.483 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.481+0000 7fcc7a738640 1 -- 172.21.15.163:0/873808824 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc74140880 con 0x7fcc740d2170 2024-05-08T06:37:05.484 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.482+0000 7fcc7a738640 1 -- 172.21.15.163:0/873808824 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc741401b0 con 0x7fcc740d2170 2024-05-08T06:37:05.484 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.482+0000 7fcc70ff9640 1 -- 172.21.15.163:0/873808824 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcc6801a070 con 0x7fcc740d2170 2024-05-08T06:37:05.484 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.483+0000 7fcc70ff9640 1 --2- 172.21.15.163:0/873808824 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fcc600762d0 0x7fcc60078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:05.485 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.483+0000 7fcc70ff9640 1 -- 172.21.15.163:0/873808824 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcc68097590 con 0x7fcc740d2170 2024-05-08T06:37:05.485 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.483+0000 7fcc73fff640 1 --2- 172.21.15.163:0/873808824 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fcc600762d0 0x7fcc60078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:05.485 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.483+0000 7fcc73fff640 1 --2- 172.21.15.163:0/873808824 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fcc600762d0 0x7fcc60078790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcc64034000 tx=0x7fcc6402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:05.486 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.487+0000 7fcc70ff9640 1 -- 172.21.15.163:0/873808824 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcc741401b0 con 0x7fcc740d2170 2024-05-08T06:37:05.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.671+0000 7fcc7a738640 1 -- 172.21.15.163:0/873808824 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc74160b40 con 0x7fcc740d2170 2024-05-08T06:37:05.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.673+0000 7fcc70ff9640 1 -- 172.21.15.163:0/873808824 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7fcc741f9130 con 0x7fcc740d2170 2024-05-08T06:37:05.673 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:05.673 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:36:42.641260+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:36:38.853461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:36:39.952926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:37:05.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.675+0000 7fcc7a738640 1 -- 172.21.15.163:0/873808824 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fcc600762d0 msgr2=0x7fcc60078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:05.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.675+0000 7fcc7a738640 1 --2- 172.21.15.163:0/873808824 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fcc600762d0 0x7fcc60078790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcc64034000 tx=0x7fcc6402e000 comp rx=0 tx=0).stop 2024-05-08T06:37:05.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.675+0000 7fcc7a738640 1 -- 172.21.15.163:0/873808824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc740d2170 msgr2=0x7fcc741f8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:05.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.675+0000 7fcc7a738640 1 --2- 172.21.15.163:0/873808824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc740d2170 0x7fcc741f8bf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800e9b0 tx=0x7fcc6800ee70 comp rx=0 tx=0).stop 2024-05-08T06:37:05.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.675+0000 7fcc7a738640 1 -- 172.21.15.163:0/873808824 shutdown_connections 2024-05-08T06:37:05.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.675+0000 7fcc7a738640 1 --2- 172.21.15.163:0/873808824 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fcc600762d0 0x7fcc60078790 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.675+0000 7fcc7a738640 1 --2- 172.21.15.163:0/873808824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc74162900 0x7fcc74145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.675+0000 7fcc7a738640 1 --2- 172.21.15.163:0/873808824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc740d2170 0x7fcc741f8bf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.675+0000 7fcc7a738640 1 --2- 172.21.15.163:0/873808824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc740aef80 0x7fcc74143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.675+0000 7fcc7a738640 1 -- 172.21.15.163:0/873808824 >> 172.21.15.163:0/873808824 conn(0x7fcc7406d610 msgr2=0x7fcc740b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:05.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.675+0000 7fcc7a738640 1 -- 172.21.15.163:0/873808824 shutdown_connections 2024-05-08T06:37:05.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.675+0000 7fcc7a738640 1 -- 172.21.15.163:0/873808824 wait complete. 2024-05-08T06:37:05.686 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T06:36:30.111982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T06:36:38.853461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T06:36:39.952926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T06:37:05.687 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T06:37:05.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.791+0000 7f1325f5f640 1 Processor -- start 2024-05-08T06:37:05.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.792+0000 7f1325f5f640 1 -- start start 2024-05-08T06:37:05.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.792+0000 7f1325f5f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f132015ba10 0x7f13200ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:05.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.792+0000 7f1325f5f640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13200b6290 con 0x7f13200b5ac0 2024-05-08T06:37:05.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.792+0000 7f1325f5f640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13200b5fd0 con 0x7f132015f3d0 2024-05-08T06:37:05.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.792+0000 7f1325f5f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f132006e5e0 con 0x7f132015ba10 2024-05-08T06:37:05.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.792+0000 7f131ffff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f13200b5ac0 0x7f1320072510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45866/0 (socket says 172.21.15.163:45866) 2024-05-08T06:37:05.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131ffff640 1 -- 172.21.15.163:0/2256704951 learned_addr learned my addr 172.21.15.163:0/2256704951 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:05.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 -- 172.21.15.163:0/2256704951 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 256431654 0 0) 0x7f13200b6290 con 0x7f13200b5ac0 2024-05-08T06:37:05.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131effd640 1 --2- 172.21.15.163:0/2256704951 >> v2:172.21.15.192:3300/0 conn(0x7f132015ba10 0x7f13200ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:05.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 -- 172.21.15.163:0/2256704951 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1308003520 con 0x7f13200b5ac0 2024-05-08T06:37:05.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 -- 172.21.15.163:0/2256704951 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3282545733 0 0) 0x7f1308003520 con 0x7f13200b5ac0 2024-05-08T06:37:05.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 -- 172.21.15.163:0/2256704951 >> v2:172.21.15.192:3300/0 conn(0x7f132015ba10 msgr2=0x7f13200ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:05.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 --2- 172.21.15.163:0/2256704951 >> v2:172.21.15.192:3300/0 conn(0x7f132015ba10 0x7f13200ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 -- 172.21.15.163:0/2256704951 >> v1:172.21.15.192:6789/0 conn(0x7f132015f3d0 legacy=0x7f132015f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:05.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 -- 172.21.15.163:0/2256704951 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132006e8c0 con 0x7f13200b5ac0 2024-05-08T06:37:05.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 -- 172.21.15.163:0/2256704951 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f1318001710 con 0x7f13200b5ac0 2024-05-08T06:37:05.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 -- 172.21.15.163:0/2256704951 >> v1:172.21.15.163:6789/0 conn(0x7f13200b5ac0 legacy=0x7f1320072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:05.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 --2- 172.21.15.163:0/2256704951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1308003a50 0x7f1308003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 --2- 172.21.15.163:0/2256704951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1308004800 0x7f1308004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 --2- 172.21.15.163:0/2256704951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13080053c0 0x7f130800cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 -- 172.21.15.163:0/2256704951 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f132006e5e0 con 0x7f13080053c0 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 -- 172.21.15.163:0/2256704951 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f130800d340 con 0x7f1308004800 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131e7fc640 1 -- 172.21.15.163:0/2256704951 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f130800d540 con 0x7f1308003a50 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131ffff640 1 --2- 172.21.15.163:0/2256704951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13080053c0 0x7f130800cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131effd640 1 --2- 172.21.15.163:0/2256704951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1308004800 0x7f1308004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131f7fe640 1 --2- 172.21.15.163:0/2256704951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1308003a50 0x7f1308003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131ffff640 1 -- 172.21.15.163:0/2256704951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1308003a50 msgr2=0x7f1308003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131ffff640 1 --2- 172.21.15.163:0/2256704951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1308003a50 0x7f1308003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131ffff640 1 -- 172.21.15.163:0/2256704951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1308004800 msgr2=0x7f1308004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.793+0000 7f131ffff640 1 --2- 172.21.15.163:0/2256704951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1308004800 0x7f1308004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.794+0000 7f131ffff640 1 -- 172.21.15.163:0/2256704951 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1308003520 con 0x7f13080053c0 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.794+0000 7f131effd640 1 --2- 172.21.15.163:0/2256704951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1308004800 0x7f1308004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:05.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.794+0000 7f131ffff640 1 -- 172.21.15.163:0/2256704951 reap_dead start 2024-05-08T06:37:05.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.794+0000 7f131ffff640 1 --2- 172.21.15.163:0/2256704951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13080053c0 0x7f130800cad0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f13200d1000 tx=0x7f1308005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c87c9941b7f28a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:05.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.794+0000 7f131e7fc640 1 -- 172.21.15.163:0/2256704951 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314009690 con 0x7f13080053c0 2024-05-08T06:37:05.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.794+0000 7f131e7fc640 1 -- 172.21.15.163:0/2256704951 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1318005ad0 con 0x7f13080053c0 2024-05-08T06:37:05.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.794+0000 7f131e7fc640 1 -- 172.21.15.163:0/2256704951 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318007d10 con 0x7f13080053c0 2024-05-08T06:37:05.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.795+0000 7f1325f5f640 1 -- 172.21.15.163:0/2256704951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13080053c0 msgr2=0x7f130800cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:05.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.795+0000 7f1325f5f640 1 --2- 172.21.15.163:0/2256704951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13080053c0 0x7f130800cad0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f13200d1000 tx=0x7f1308005830 comp rx=0 tx=0).stop 2024-05-08T06:37:05.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.795+0000 7f1325f5f640 1 -- 172.21.15.163:0/2256704951 shutdown_connections 2024-05-08T06:37:05.799 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.795+0000 7f1325f5f640 1 --2- 172.21.15.163:0/2256704951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13080053c0 0x7f130800cad0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.799 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.795+0000 7f1325f5f640 1 -- 172.21.15.163:0/2256704951 >> 172.21.15.163:0/2256704951 conn(0x7f132006c500 msgr2=0x7f132006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:05.799 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.795+0000 7f1325f5f640 1 -- 172.21.15.163:0/2256704951 shutdown_connections 2024-05-08T06:37:05.799 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.795+0000 7f1325f5f640 1 -- 172.21.15.163:0/2256704951 wait complete. 2024-05-08T06:37:05.799 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.796+0000 7f1325f5f640 1 Processor -- start 2024-05-08T06:37:05.799 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.796+0000 7f1325f5f640 1 -- start start 2024-05-08T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f1325f5f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13080053c0 0x7f1320151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f1325f5f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f132009e900 0x7f132009ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f1325f5f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13201f0e40 0x7f13201f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f1325f5f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1320072690 con 0x7f13080053c0 2024-05-08T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f1325f5f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1320072510 con 0x7f13201f0e40 2024-05-08T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f1325f5f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1320072810 con 0x7f132009e900 2024-05-08T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13080053c0 0x7f1320151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13080053c0 0x7f1320151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41408/0 (socket says 172.21.15.163:41408) 2024-05-08T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f131f7fe640 1 -- 172.21.15.163:0/3206006587 learned_addr learned my addr 172.21.15.163:0/3206006587 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f131effd640 1 --2- 172.21.15.163:0/3206006587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f132009e900 0x7f132009ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f131ffff640 1 --2- 172.21.15.163:0/3206006587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13201f0e40 0x7f13201f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f131f7fe640 1 -- 172.21.15.163:0/3206006587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f132009e900 msgr2=0x7f132009ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f131f7fe640 1 --2- 172.21.15.163:0/3206006587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f132009e900 0x7f132009ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f131f7fe640 1 -- 172.21.15.163:0/3206006587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13201f0e40 msgr2=0x7f13201f12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:05.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f131f7fe640 1 --2- 172.21.15.163:0/3206006587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13201f0e40 0x7f13201f12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f131f7fe640 1 -- 172.21.15.163:0/3206006587 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13201f1800 con 0x7f13080053c0 2024-05-08T06:37:05.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.797+0000 7f131f7fe640 1 --2- 172.21.15.163:0/3206006587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13080053c0 0x7f1320151330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1310011cf0 tx=0x7f131000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:05.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.798+0000 7f1307fff640 1 -- 172.21.15.163:0/3206006587 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1310006d50 con 0x7f13080053c0 2024-05-08T06:37:05.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.798+0000 7f1307fff640 1 -- 172.21.15.163:0/3206006587 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1310003970 con 0x7f13080053c0 2024-05-08T06:37:05.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.798+0000 7f1307fff640 1 -- 172.21.15.163:0/3206006587 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1310003d40 con 0x7f13080053c0 2024-05-08T06:37:05.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.798+0000 7f1325f5f640 1 -- 172.21.15.163:0/3206006587 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132009d860 con 0x7f13080053c0 2024-05-08T06:37:05.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.798+0000 7f1325f5f640 1 -- 172.21.15.163:0/3206006587 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132009de20 con 0x7f13080053c0 2024-05-08T06:37:05.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.799+0000 7f1325f5f640 1 -- 172.21.15.163:0/3206006587 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f132006a550 con 0x7f13080053c0 2024-05-08T06:37:05.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.799+0000 7f1307fff640 1 -- 172.21.15.163:0/3206006587 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f131000c030 con 0x7f13080053c0 2024-05-08T06:37:05.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.800+0000 7f1307fff640 1 --2- 172.21.15.163:0/3206006587 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f130c076300 0x7f130c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:05.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.800+0000 7f1307fff640 1 -- 172.21.15.163:0/3206006587 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f131009a5a0 con 0x7f13080053c0 2024-05-08T06:37:05.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.800+0000 7f131effd640 1 --2- 172.21.15.163:0/3206006587 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f130c076300 0x7f130c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:05.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.800+0000 7f131effd640 1 --2- 172.21.15.163:0/3206006587 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f130c076300 0x7f130c0787c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f13201f4310 tx=0x7f1314008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:05.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.804+0000 7f1307fff640 1 -- 172.21.15.163:0/3206006587 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f13201f1800 con 0x7f13080053c0 2024-05-08T06:37:05.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.985+0000 7f1325f5f640 1 -- 172.21.15.163:0/3206006587 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f132006caf0 con 0x7f13080053c0 2024-05-08T06:37:05.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.986+0000 7f1307fff640 1 -- 172.21.15.163:0/3206006587 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f1310063e70 con 0x7f13080053c0 2024-05-08T06:37:05.985 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2024-05-08T06:37:05.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.988+0000 7f1325f5f640 1 -- 172.21.15.163:0/3206006587 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f130c076300 msgr2=0x7f130c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:05.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.988+0000 7f1325f5f640 1 --2- 172.21.15.163:0/3206006587 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f130c076300 0x7f130c0787c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f13201f4310 tx=0x7f1314008040 comp rx=0 tx=0).stop 2024-05-08T06:37:05.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.988+0000 7f1325f5f640 1 -- 172.21.15.163:0/3206006587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13080053c0 msgr2=0x7f1320151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:05.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.988+0000 7f1325f5f640 1 --2- 172.21.15.163:0/3206006587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13080053c0 0x7f1320151330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1310011cf0 tx=0x7f131000f730 comp rx=0 tx=0).stop 2024-05-08T06:37:05.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.988+0000 7f1325f5f640 1 -- 172.21.15.163:0/3206006587 shutdown_connections 2024-05-08T06:37:05.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.988+0000 7f1325f5f640 1 --2- 172.21.15.163:0/3206006587 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f130c076300 0x7f130c0787c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.988+0000 7f1325f5f640 1 --2- 172.21.15.163:0/3206006587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13201f0e40 0x7f13201f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.988+0000 7f1325f5f640 1 --2- 172.21.15.163:0/3206006587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f132009e900 0x7f132009ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.988+0000 7f1325f5f640 1 --2- 172.21.15.163:0/3206006587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13080053c0 0x7f1320151330 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:05.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.988+0000 7f1325f5f640 1 -- 172.21.15.163:0/3206006587 >> 172.21.15.163:0/3206006587 conn(0x7f132006c500 msgr2=0x7f13200b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:05.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.988+0000 7f1325f5f640 1 -- 172.21.15.163:0/3206006587 shutdown_connections 2024-05-08T06:37:05.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:05.988+0000 7f1325f5f640 1 -- 172.21.15.163:0/3206006587 wait complete. 2024-05-08T06:37:05.999 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T06:37:06.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.103+0000 7ff7b66d1640 1 Processor -- start 2024-05-08T06:37:06.103 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.103+0000 7ff7b66d1640 1 -- start start 2024-05-08T06:37:06.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.104+0000 7ff7b66d1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7b01563d0 0x7ff7b0160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.104+0000 7ff7b66d1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7b00b12b0 con 0x7ff7b00b0db0 2024-05-08T06:37:06.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.104+0000 7ff7b66d1640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7b00b1590 con 0x7ff7b00d2180 2024-05-08T06:37:06.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.104+0000 7ff7b66d1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b015fe70 con 0x7ff7b01563d0 2024-05-08T06:37:06.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.104+0000 7ff7af7fe640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff7b00b0db0 0x7ff7b0073d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45874/0 (socket says 172.21.15.163:45874) 2024-05-08T06:37:06.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.104+0000 7ff7af7fe640 1 -- 172.21.15.163:0/2720714724 learned_addr learned my addr 172.21.15.163:0/2720714724 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:06.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.104+0000 7ff7affff640 1 --2- 172.21.15.163:0/2720714724 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7b01563d0 0x7ff7b0160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.104+0000 7ff7aeffd640 1 -- 172.21.15.163:0/2720714724 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689489296 0 0) 0x7ff7b00b12b0 con 0x7ff7b00b0db0 2024-05-08T06:37:06.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.104+0000 7ff7aeffd640 1 -- 172.21.15.163:0/2720714724 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff798003520 con 0x7ff7b00b0db0 2024-05-08T06:37:06.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7aeffd640 1 -- 172.21.15.163:0/2720714724 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 904952655 0 0) 0x7ff798003520 con 0x7ff7b00b0db0 2024-05-08T06:37:06.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7aeffd640 1 -- 172.21.15.163:0/2720714724 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7b01563d0 msgr2=0x7ff7b0160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:06.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7aeffd640 1 --2- 172.21.15.163:0/2720714724 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7b01563d0 0x7ff7b0160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7aeffd640 1 -- 172.21.15.163:0/2720714724 >> v1:172.21.15.192:6789/0 conn(0x7ff7b00d2180 legacy=0x7ff7b0162fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7aeffd640 1 -- 172.21.15.163:0/2720714724 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b015fb00 con 0x7ff7b00b0db0 2024-05-08T06:37:06.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7aeffd640 1 -- 172.21.15.163:0/2720714724 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7ff7a40016f0 con 0x7ff7b00b0db0 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7aeffd640 1 -- 172.21.15.163:0/2720714724 >> v1:172.21.15.163:6789/0 conn(0x7ff7b00b0db0 legacy=0x7ff7b0073d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7aeffd640 1 --2- 172.21.15.163:0/2720714724 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7b01563d0 0x7ff7980039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7aeffd640 1 --2- 172.21.15.163:0/2720714724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff798003f70 0x7ff798004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7aeffd640 1 --2- 172.21.15.163:0/2720714724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff798004d40 0x7ff7980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7aeffd640 1 -- 172.21.15.163:0/2720714724 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b015fe70 con 0x7ff798003f70 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7aeffd640 1 -- 172.21.15.163:0/2720714724 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff79800bad0 con 0x7ff798004d40 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7aeffd640 1 -- 172.21.15.163:0/2720714724 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff79800bcd0 con 0x7ff7b01563d0 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7af7fe640 1 --2- 172.21.15.163:0/2720714724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff798003f70 0x7ff798004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7affff640 1 --2- 172.21.15.163:0/2720714724 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7b01563d0 0x7ff7980039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7b4c6e640 1 --2- 172.21.15.163:0/2720714724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff798004d40 0x7ff7980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7af7fe640 1 -- 172.21.15.163:0/2720714724 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7b01563d0 msgr2=0x7ff7980039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7af7fe640 1 --2- 172.21.15.163:0/2720714724 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7b01563d0 0x7ff7980039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7af7fe640 1 -- 172.21.15.163:0/2720714724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff798004d40 msgr2=0x7ff7980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7af7fe640 1 --2- 172.21.15.163:0/2720714724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff798004d40 0x7ff7980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.105+0000 7ff7af7fe640 1 -- 172.21.15.163:0/2720714724 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff798003520 con 0x7ff798003f70 2024-05-08T06:37:06.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.106+0000 7ff7af7fe640 1 --2- 172.21.15.163:0/2720714724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff798003f70 0x7ff798004800 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4001eb0 tx=0x7ff798005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ed7227640ea304 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:06.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.106+0000 7ff7aeffd640 1 -- 172.21.15.163:0/2720714724 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4007d30 con 0x7ff798003f70 2024-05-08T06:37:06.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.106+0000 7ff7aeffd640 1 -- 172.21.15.163:0/2720714724 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a4005af0 con 0x7ff798003f70 2024-05-08T06:37:06.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.106+0000 7ff7aeffd640 1 -- 172.21.15.163:0/2720714724 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4007070 con 0x7ff798003f70 2024-05-08T06:37:06.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.107+0000 7ff7b66d1640 1 -- 172.21.15.163:0/2720714724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff798003f70 msgr2=0x7ff798004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.107+0000 7ff7b66d1640 1 --2- 172.21.15.163:0/2720714724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff798003f70 0x7ff798004800 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4001eb0 tx=0x7ff798005830 comp rx=0 tx=0).stop 2024-05-08T06:37:06.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.107+0000 7ff7b4c6e640 1 -- 172.21.15.163:0/2720714724 reap_dead start 2024-05-08T06:37:06.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.107+0000 7ff7b66d1640 1 -- 172.21.15.163:0/2720714724 shutdown_connections 2024-05-08T06:37:06.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.107+0000 7ff7b66d1640 1 -- 172.21.15.163:0/2720714724 >> 172.21.15.163:0/2720714724 conn(0x7ff7b006d610 msgr2=0x7ff7b006da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:06.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.107+0000 7ff7b66d1640 1 -- 172.21.15.163:0/2720714724 shutdown_connections 2024-05-08T06:37:06.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.107+0000 7ff7b66d1640 1 -- 172.21.15.163:0/2720714724 wait complete. 2024-05-08T06:37:06.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.108+0000 7ff7b66d1640 1 Processor -- start 2024-05-08T06:37:06.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.108+0000 7ff7b66d1640 1 -- start start 2024-05-08T06:37:06.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7b66d1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff798003f70 0x7ff7b00b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7b66d1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7b0147870 0x7ff7b00aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7b66d1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7b0147cd0 0x7ff7b00af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7b66d1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0156930 con 0x7ff798003f70 2024-05-08T06:37:06.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7b66d1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01567b0 con 0x7ff7b0147870 2024-05-08T06:37:06.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7b66d1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b01636f0 con 0x7ff7b0147cd0 2024-05-08T06:37:06.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7affff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff798003f70 0x7ff7b00b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7affff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff798003f70 0x7ff7b00b2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41422/0 (socket says 172.21.15.163:41422) 2024-05-08T06:37:06.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7affff640 1 -- 172.21.15.163:0/1489904278 learned_addr learned my addr 172.21.15.163:0/1489904278 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:06.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7af7fe640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7b0147870 0x7ff7b00aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7b4c6e640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7b0147cd0 0x7ff7b00af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7affff640 1 -- 172.21.15.163:0/1489904278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7b0147cd0 msgr2=0x7ff7b00af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7affff640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7b0147cd0 0x7ff7b00af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7affff640 1 -- 172.21.15.163:0/1489904278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7b0147870 msgr2=0x7ff7b00aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7affff640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7b0147870 0x7ff7b00aeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7affff640 1 -- 172.21.15.163:0/1489904278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b00b1b90 con 0x7ff798003f70 2024-05-08T06:37:06.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7af7fe640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7b0147870 0x7ff7b00aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:06.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7b4c6e640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7b0147cd0 0x7ff7b00af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:06.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7affff640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff798003f70 0x7ff7b00b2c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00119a0 tx=0x7ff7a0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:06.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7acff9640 1 -- 172.21.15.163:0/1489904278 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000faf0 con 0x7ff798003f70 2024-05-08T06:37:06.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7acff9640 1 -- 172.21.15.163:0/1489904278 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a000fc90 con 0x7ff798003f70 2024-05-08T06:37:06.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7b66d1640 1 -- 172.21.15.163:0/1489904278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b00b1e70 con 0x7ff798003f70 2024-05-08T06:37:06.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.109+0000 7ff7acff9640 1 -- 172.21.15.163:0/1489904278 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0014440 con 0x7ff798003f70 2024-05-08T06:37:06.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.110+0000 7ff7b66d1640 1 -- 172.21.15.163:0/1489904278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b00b2430 con 0x7ff798003f70 2024-05-08T06:37:06.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.111+0000 7ff7b66d1640 1 -- 172.21.15.163:0/1489904278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b006a550 con 0x7ff798003f70 2024-05-08T06:37:06.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.111+0000 7ff7acff9640 1 -- 172.21.15.163:0/1489904278 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff7a0012040 con 0x7ff798003f70 2024-05-08T06:37:06.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.112+0000 7ff7acff9640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff79c0762d0 0x7ff79c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.112+0000 7ff7acff9640 1 -- 172.21.15.163:0/1489904278 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff7a0097ca0 con 0x7ff798003f70 2024-05-08T06:37:06.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.112+0000 7ff7af7fe640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff79c0762d0 0x7ff79c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.112+0000 7ff7af7fe640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff79c0762d0 0x7ff79c078790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4032000 tx=0x7ff7a402d000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:06.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.116+0000 7ff7acff9640 1 -- 172.21.15.163:0/1489904278 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff7a0019030 con 0x7ff798003f70 2024-05-08T06:37:06.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.298+0000 7ff7b66d1640 1 -- 172.21.15.163:0/1489904278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff7b00d12a0 con 0x7ff798003f70 2024-05-08T06:37:06.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.298+0000 7ff7acff9640 1 -- 172.21.15.163:0/1489904278 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ff7a0061600 con 0x7ff798003f70 2024-05-08T06:37:06.297 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2024-05-08T06:37:06.299 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.300+0000 7ff7b66d1640 1 -- 172.21.15.163:0/1489904278 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff79c0762d0 msgr2=0x7ff79c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.300+0000 7ff7b66d1640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff79c0762d0 0x7ff79c078790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4032000 tx=0x7ff7a402d000 comp rx=0 tx=0).stop 2024-05-08T06:37:06.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.300+0000 7ff7b66d1640 1 -- 172.21.15.163:0/1489904278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff798003f70 msgr2=0x7ff7b00b2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.300+0000 7ff7b66d1640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff798003f70 0x7ff7b00b2c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00119a0 tx=0x7ff7a0011e60 comp rx=0 tx=0).stop 2024-05-08T06:37:06.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.300+0000 7ff7b66d1640 1 -- 172.21.15.163:0/1489904278 shutdown_connections 2024-05-08T06:37:06.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.300+0000 7ff7b66d1640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff79c0762d0 0x7ff79c078790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.300+0000 7ff7b66d1640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7b0147cd0 0x7ff7b00af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.300+0000 7ff7b66d1640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7b0147870 0x7ff7b00aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.300+0000 7ff7b66d1640 1 --2- 172.21.15.163:0/1489904278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff798003f70 0x7ff7b00b2c30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.300+0000 7ff7b66d1640 1 -- 172.21.15.163:0/1489904278 >> 172.21.15.163:0/1489904278 conn(0x7ff7b006d610 msgr2=0x7ff7b0162310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:06.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.300+0000 7ff7b66d1640 1 -- 172.21.15.163:0/1489904278 shutdown_connections 2024-05-08T06:37:06.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.300+0000 7ff7b66d1640 1 -- 172.21.15.163:0/1489904278 wait complete. 2024-05-08T06:37:06.311 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T06:37:06.414 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.414+0000 7fb5eedf2640 1 Processor -- start 2024-05-08T06:37:06.414 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.415+0000 7fb5eedf2640 1 -- start start 2024-05-08T06:37:06.415 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.415+0000 7fb5eedf2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb5e8163960 0x7fb5e80d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.416 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.415+0000 7fb5eedf2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e8163f00 con 0x7fb5e80d20d0 2024-05-08T06:37:06.416 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.415+0000 7fb5eedf2640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e80cfe20 con 0x7fb5e815f5a0 2024-05-08T06:37:06.416 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.415+0000 7fb5eedf2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fb5e80d0100 con 0x7fb5e8163960 2024-05-08T06:37:06.416 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.415+0000 7fb5ed38f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb5e8163960 0x7fb5e80d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.416 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.415+0000 7fb5e7fff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb5e80d20d0 0x7fb5e8163250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45878/0 (socket says 172.21.15.163:45878) 2024-05-08T06:37:06.416 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.415+0000 7fb5e7fff640 1 -- 172.21.15.163:0/1727721287 learned_addr learned my addr 172.21.15.163:0/1727721287 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:06.416 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.416+0000 7fb5ed38f640 1 -- 172.21.15.163:0/1727721287 >> v1:172.21.15.192:6789/0 conn(0x7fb5e815f5a0 legacy=0x7fb5e80d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:06.416 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.416+0000 7fb5ed38f640 1 -- 172.21.15.163:0/1727721287 >> v1:172.21.15.163:6789/0 conn(0x7fb5e80d20d0 legacy=0x7fb5e8163250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.416 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.416+0000 7fb5e77fe640 1 -- 172.21.15.163:0/1727721287 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1264469017 0 0) 0x7fb5e8163f00 con 0x7fb5e80d20d0 2024-05-08T06:37:06.416 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.416+0000 7fb5ed38f640 1 -- 172.21.15.163:0/1727721287 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e80d03e0 con 0x7fb5e8163960 2024-05-08T06:37:06.416 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.416+0000 7fb5ed38f640 1 --2- 172.21.15.163:0/1727721287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5e8163960 0x7fb5e80d0900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb5e00097a0 tx=0x7fb5e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66d03140418e818 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:06.416 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.417+0000 7fb5e77fe640 1 -- 172.21.15.163:0/1727721287 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0037070 con 0x7fb5e8163960 2024-05-08T06:37:06.416 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.417+0000 7fb5e77fe640 1 -- 172.21.15.163:0/1727721287 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e0032440 con 0x7fb5e8163960 2024-05-08T06:37:06.417 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.417+0000 7fb5e77fe640 1 -- 172.21.15.163:0/1727721287 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0032810 con 0x7fb5e8163960 2024-05-08T06:37:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.417+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1727721287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5e8163960 msgr2=0x7fb5e80d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.417+0000 7fb5eedf2640 1 --2- 172.21.15.163:0/1727721287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5e8163960 0x7fb5e80d0900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb5e00097a0 tx=0x7fb5e002a180 comp rx=0 tx=0).stop 2024-05-08T06:37:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.417+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1727721287 shutdown_connections 2024-05-08T06:37:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.417+0000 7fb5eedf2640 1 --2- 172.21.15.163:0/1727721287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5e8163960 0x7fb5e80d0900 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.417+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1727721287 >> 172.21.15.163:0/1727721287 conn(0x7fb5e8074080 msgr2=0x7fb5e806d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.417+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1727721287 shutdown_connections 2024-05-08T06:37:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.417+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1727721287 wait complete. 2024-05-08T06:37:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.418+0000 7fb5eedf2640 1 Processor -- start 2024-05-08T06:37:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.418+0000 7fb5eedf2640 1 -- start start 2024-05-08T06:37:06.420 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.419+0000 7fb5eedf2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5e80d20d0 0x7fb5e81505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.420 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.419+0000 7fb5eedf2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5e815f5a0 0x7fb5e80c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.420 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.419+0000 7fb5eedf2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5e8163960 0x7fb5e81f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.420 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.419+0000 7fb5eedf2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e80d38e0 con 0x7fb5e815f5a0 2024-05-08T06:37:06.420 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.419+0000 7fb5eedf2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e815fb90 con 0x7fb5e8163960 2024-05-08T06:37:06.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.419+0000 7fb5eedf2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e80d3a60 con 0x7fb5e80d20d0 2024-05-08T06:37:06.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.419+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5e815f5a0 0x7fb5e80c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.419+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5e815f5a0 0x7fb5e80c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41440/0 (socket says 172.21.15.163:41440) 2024-05-08T06:37:06.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.419+0000 7fb5e7fff640 1 -- 172.21.15.163:0/1362462913 learned_addr learned my addr 172.21.15.163:0/1362462913 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:06.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.420+0000 7fb5ecb8e640 1 --2- 172.21.15.163:0/1362462913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5e80d20d0 0x7fb5e81505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.420+0000 7fb5ed38f640 1 --2- 172.21.15.163:0/1362462913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5e8163960 0x7fb5e81f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.420+0000 7fb5e7fff640 1 -- 172.21.15.163:0/1362462913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5e80d20d0 msgr2=0x7fb5e81505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.420+0000 7fb5e7fff640 1 --2- 172.21.15.163:0/1362462913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5e80d20d0 0x7fb5e81505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.420+0000 7fb5e7fff640 1 -- 172.21.15.163:0/1362462913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5e8163960 msgr2=0x7fb5e81f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.420+0000 7fb5e7fff640 1 --2- 172.21.15.163:0/1362462913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5e8163960 0x7fb5e81f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.420+0000 7fb5e7fff640 1 -- 172.21.15.163:0/1362462913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e81f9490 con 0x7fb5e815f5a0 2024-05-08T06:37:06.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.420+0000 7fb5e7fff640 1 --2- 172.21.15.163:0/1362462913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5e815f5a0 0x7fb5e80c98a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00c4c0 tx=0x7fb5dc00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:06.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.420+0000 7fb5e57fa640 1 -- 172.21.15.163:0/1362462913 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc018070 con 0x7fb5e815f5a0 2024-05-08T06:37:06.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.420+0000 7fb5e57fa640 1 -- 172.21.15.163:0/1362462913 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5dc007500 con 0x7fb5e815f5a0 2024-05-08T06:37:06.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.420+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1362462913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e81f9770 con 0x7fb5e815f5a0 2024-05-08T06:37:06.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.420+0000 7fb5e57fa640 1 -- 172.21.15.163:0/1362462913 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc004a20 con 0x7fb5e815f5a0 2024-05-08T06:37:06.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.420+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1362462913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e80c90a0 con 0x7fb5e815f5a0 2024-05-08T06:37:06.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.421+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1362462913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e806a550 con 0x7fb5e815f5a0 2024-05-08T06:37:06.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.422+0000 7fb5e57fa640 1 -- 172.21.15.163:0/1362462913 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb5dc01b070 con 0x7fb5e815f5a0 2024-05-08T06:37:06.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.422+0000 7fb5e57fa640 1 --2- 172.21.15.163:0/1362462913 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb5d40762d0 0x7fb5d4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.422+0000 7fb5e57fa640 1 -- 172.21.15.163:0/1362462913 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb5dc096840 con 0x7fb5e815f5a0 2024-05-08T06:37:06.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.422+0000 7fb5ecb8e640 1 --2- 172.21.15.163:0/1362462913 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb5d40762d0 0x7fb5d4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.423+0000 7fb5ecb8e640 1 --2- 172.21.15.163:0/1362462913 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb5d40762d0 0x7fb5d4078790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8005520 tx=0x7fb5d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:06.425 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.426+0000 7fb5e57fa640 1 -- 172.21.15.163:0/1362462913 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb5dc0601a0 con 0x7fb5e815f5a0 2024-05-08T06:37:06.608 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.609+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1362462913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb5e80d3170 con 0x7fb5e815f5a0 2024-05-08T06:37:06.608 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.609+0000 7fb5e57fa640 1 -- 172.21.15.163:0/1362462913 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fb5dc068190 con 0x7fb5e815f5a0 2024-05-08T06:37:06.609 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2024-05-08T06:37:06.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.611+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1362462913 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb5d40762d0 msgr2=0x7fb5d4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.611+0000 7fb5eedf2640 1 --2- 172.21.15.163:0/1362462913 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb5d40762d0 0x7fb5d4078790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8005520 tx=0x7fb5d8009040 comp rx=0 tx=0).stop 2024-05-08T06:37:06.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.611+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1362462913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5e815f5a0 msgr2=0x7fb5e80c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.611+0000 7fb5eedf2640 1 --2- 172.21.15.163:0/1362462913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5e815f5a0 0x7fb5e80c98a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00c4c0 tx=0x7fb5dc00c980 comp rx=0 tx=0).stop 2024-05-08T06:37:06.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.611+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1362462913 shutdown_connections 2024-05-08T06:37:06.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.611+0000 7fb5eedf2640 1 --2- 172.21.15.163:0/1362462913 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb5d40762d0 0x7fb5d4078790 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.611+0000 7fb5eedf2640 1 --2- 172.21.15.163:0/1362462913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5e8163960 0x7fb5e81f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.611+0000 7fb5eedf2640 1 --2- 172.21.15.163:0/1362462913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5e815f5a0 0x7fb5e80c98a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.611+0000 7fb5eedf2640 1 --2- 172.21.15.163:0/1362462913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5e80d20d0 0x7fb5e81505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.611+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1362462913 >> 172.21.15.163:0/1362462913 conn(0x7fb5e8074080 msgr2=0x7fb5e8160010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:06.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.611+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1362462913 shutdown_connections 2024-05-08T06:37:06.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.612+0000 7fb5eedf2640 1 -- 172.21.15.163:0/1362462913 wait complete. 2024-05-08T06:37:06.622 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_flush.TestFlush.test_flush' 2024-05-08T06:37:06.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.723+0000 7fe56ab5f640 1 Processor -- start 2024-05-08T06:37:06.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.723+0000 7fe56ab5f640 1 -- start start 2024-05-08T06:37:06.724 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe56ab5f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe56415f3e0 0x7fe56406fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.724 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe56ab5f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56415f7c0 0x7fe5640cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.724 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe56ab5f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe5640b3c30 0x7fe5640b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.724 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe56ab5f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fe5640cfe90 con 0x7fe56415f3e0 2024-05-08T06:37:06.724 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe56ab5f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe5640d08a0 con 0x7fe5640b3c30 2024-05-08T06:37:06.724 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe56ab5f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe5640d00e0 con 0x7fe56415f7c0 2024-05-08T06:37:06.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe5688fb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe56415f3e0 0x7fe56406fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe5688fb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe56415f3e0 0x7fe56406fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41454/0 (socket says 172.21.15.163:41454) 2024-05-08T06:37:06.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe5688fb640 1 -- 172.21.15.163:0/2442497248 learned_addr learned my addr 172.21.15.163:0/2442497248 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:06.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe55bfff640 1 --2- 172.21.15.163:0/2442497248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56415f7c0 0x7fe5640cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe5688fb640 1 -- 172.21.15.163:0/2442497248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56415f7c0 msgr2=0x7fe5640cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:06.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe5690fc640 1 --2- 172.21.15.163:0/2442497248 >> v2:172.21.15.192:3300/0 conn(0x7fe5640b3c30 0x7fe5640b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe5688fb640 1 --2- 172.21.15.163:0/2442497248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56415f7c0 0x7fe5640cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe5688fb640 1 -- 172.21.15.163:0/2442497248 >> v2:172.21.15.192:3300/0 conn(0x7fe5640b3c30 msgr2=0x7fe5640b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:06.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe5688fb640 1 --2- 172.21.15.163:0/2442497248 >> v2:172.21.15.192:3300/0 conn(0x7fe5640b3c30 0x7fe5640b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.724+0000 7fe5688fb640 1 -- 172.21.15.163:0/2442497248 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5640d03c0 con 0x7fe56415f3e0 2024-05-08T06:37:06.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.725+0000 7fe5688fb640 1 --2- 172.21.15.163:0/2442497248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe56415f3e0 0x7fe56406fa60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe5640d1f30 tx=0x7fe56002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e0e301292bc85d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:06.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.725+0000 7fe55b7fe640 1 -- 172.21.15.163:0/2442497248 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560037070 con 0x7fe56415f3e0 2024-05-08T06:37:06.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.725+0000 7fe55b7fe640 1 -- 172.21.15.163:0/2442497248 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe560032440 con 0x7fe56415f3e0 2024-05-08T06:37:06.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.725+0000 7fe55b7fe640 1 -- 172.21.15.163:0/2442497248 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560032810 con 0x7fe56415f3e0 2024-05-08T06:37:06.728 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.726+0000 7fe56ab5f640 1 -- 172.21.15.163:0/2442497248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe56415f3e0 msgr2=0x7fe56406fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.728 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.726+0000 7fe56ab5f640 1 --2- 172.21.15.163:0/2442497248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe56415f3e0 0x7fe56406fa60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe5640d1f30 tx=0x7fe56002a1e0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.728 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.726+0000 7fe56ab5f640 1 -- 172.21.15.163:0/2442497248 shutdown_connections 2024-05-08T06:37:06.728 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.726+0000 7fe56ab5f640 1 --2- 172.21.15.163:0/2442497248 >> v2:172.21.15.192:3300/0 conn(0x7fe5640b3c30 0x7fe5640b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.728 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.726+0000 7fe56ab5f640 1 --2- 172.21.15.163:0/2442497248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56415f7c0 0x7fe5640cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.726+0000 7fe56ab5f640 1 --2- 172.21.15.163:0/2442497248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe56415f3e0 0x7fe56406fa60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.726+0000 7fe56ab5f640 1 -- 172.21.15.163:0/2442497248 >> 172.21.15.163:0/2442497248 conn(0x7fe56406d430 msgr2=0x7fe56406d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.726+0000 7fe56ab5f640 1 -- 172.21.15.163:0/2442497248 shutdown_connections 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.726+0000 7fe56ab5f640 1 -- 172.21.15.163:0/2442497248 wait complete. 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.727+0000 7fe56ab5f640 1 Processor -- start 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.727+0000 7fe56ab5f640 1 -- start start 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe56ab5f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5640b3c30 0x7fe56415cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe56ab5f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56415f3e0 0x7fe5641f8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe56ab5f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe56415f7c0 0x7fe5641f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe56ab5f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe5640736b0 con 0x7fe56415f7c0 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe56ab5f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5640d0dd0 con 0x7fe5640b3c30 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe56ab5f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe564073830 con 0x7fe56415f3e0 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe5690fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe56415f7c0 0x7fe5641f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe5690fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe56415f7c0 0x7fe5641f8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41466/0 (socket says 172.21.15.163:41466) 2024-05-08T06:37:06.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe5690fc640 1 -- 172.21.15.163:0/3798267248 learned_addr learned my addr 172.21.15.163:0/3798267248 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:06.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe55bfff640 1 --2- 172.21.15.163:0/3798267248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56415f3e0 0x7fe5641f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe5688fb640 1 --2- 172.21.15.163:0/3798267248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5640b3c30 0x7fe56415cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe5690fc640 1 -- 172.21.15.163:0/3798267248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56415f3e0 msgr2=0x7fe5641f8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe5690fc640 1 --2- 172.21.15.163:0/3798267248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56415f3e0 0x7fe5641f8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe5690fc640 1 -- 172.21.15.163:0/3798267248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5640b3c30 msgr2=0x7fe56415cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:06.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe5690fc640 1 --2- 172.21.15.163:0/3798267248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5640b3c30 0x7fe56415cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:06.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe5690fc640 1 -- 172.21.15.163:0/3798267248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56415d510 con 0x7fe56415f7c0 2024-05-08T06:37:06.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe5690fc640 1 --2- 172.21.15.163:0/3798267248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe56415f7c0 0x7fe5641f8cb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00cd70 tx=0x7fe55c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:06.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe5597fa640 1 -- 172.21.15.163:0/3798267248 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c003b90 con 0x7fe56415f7c0 2024-05-08T06:37:06.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.728+0000 7fe5597fa640 1 -- 172.21.15.163:0/3798267248 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe55c003d30 con 0x7fe56415f7c0 2024-05-08T06:37:06.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.729+0000 7fe56ab5f640 1 -- 172.21.15.163:0/3798267248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56415d7f0 con 0x7fe56415f7c0 2024-05-08T06:37:06.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.729+0000 7fe5597fa640 1 -- 172.21.15.163:0/3798267248 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c004240 con 0x7fe56415f7c0 2024-05-08T06:37:06.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.729+0000 7fe56ab5f640 1 -- 172.21.15.163:0/3798267248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56415ddb0 con 0x7fe56415f7c0 2024-05-08T06:37:06.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.730+0000 7fe56ab5f640 1 -- 172.21.15.163:0/3798267248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56406a4b0 con 0x7fe56415f7c0 2024-05-08T06:37:06.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.730+0000 7fe5597fa640 1 -- 172.21.15.163:0/3798267248 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe55c0043e0 con 0x7fe56415f7c0 2024-05-08T06:37:06.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.731+0000 7fe5597fa640 1 --2- 172.21.15.163:0/3798267248 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe5500762d0 0x7fe550078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:06.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.731+0000 7fe5688fb640 1 --2- 172.21.15.163:0/3798267248 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe5500762d0 0x7fe550078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:06.731 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.731+0000 7fe5597fa640 1 -- 172.21.15.163:0/3798267248 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe55c097fc0 con 0x7fe56415f7c0 2024-05-08T06:37:06.732 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.731+0000 7fe5688fb640 1 --2- 172.21.15.163:0/3798267248 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe5500762d0 0x7fe550078790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe560034000 tx=0x7fe56002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:06.734 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.735+0000 7fe5597fa640 1 -- 172.21.15.163:0/3798267248 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe55c0612d0 con 0x7fe56415f7c0 2024-05-08T06:37:06.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:06.987+0000 7fe56ab5f640 1 -- 172.21.15.163:0/3798267248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_flush.TestFlush.test_flush"]} v 0) -- 0x7fe564159110 con 0x7fe56415f7c0 2024-05-08T06:37:07.761 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.762+0000 7fe5597fa640 1 -- 172.21.15.163:0/3798267248 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_flush.TestFlush.test_flush"]}]=0 v47) ==== 126+0+0 (secure 0 0 0) 0x7fe55c0655d0 con 0x7fe56415f7c0 2024-05-08T06:37:07.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.765+0000 7fe56ab5f640 1 -- 172.21.15.163:0/3798267248 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe5500762d0 msgr2=0x7fe550078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:07.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.765+0000 7fe56ab5f640 1 --2- 172.21.15.163:0/3798267248 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe5500762d0 0x7fe550078790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe560034000 tx=0x7fe56002e000 comp rx=0 tx=0).stop 2024-05-08T06:37:07.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.765+0000 7fe56ab5f640 1 -- 172.21.15.163:0/3798267248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe56415f7c0 msgr2=0x7fe5641f8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:07.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.765+0000 7fe56ab5f640 1 --2- 172.21.15.163:0/3798267248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe56415f7c0 0x7fe5641f8cb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00cd70 tx=0x7fe55c00a660 comp rx=0 tx=0).stop 2024-05-08T06:37:07.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.765+0000 7fe56ab5f640 1 -- 172.21.15.163:0/3798267248 shutdown_connections 2024-05-08T06:37:07.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.765+0000 7fe56ab5f640 1 --2- 172.21.15.163:0/3798267248 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe5500762d0 0x7fe550078790 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:07.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.765+0000 7fe56ab5f640 1 --2- 172.21.15.163:0/3798267248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe56415f7c0 0x7fe5641f8cb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:07.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.765+0000 7fe56ab5f640 1 --2- 172.21.15.163:0/3798267248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56415f3e0 0x7fe5641f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:07.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.765+0000 7fe56ab5f640 1 --2- 172.21.15.163:0/3798267248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5640b3c30 0x7fe56415cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:07.765 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.765+0000 7fe56ab5f640 1 -- 172.21.15.163:0/3798267248 >> 172.21.15.163:0/3798267248 conn(0x7fe56406d430 msgr2=0x7fe564158520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:07.765 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.765+0000 7fe56ab5f640 1 -- 172.21.15.163:0/3798267248 shutdown_connections 2024-05-08T06:37:07.765 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.765+0000 7fe56ab5f640 1 -- 172.21.15.163:0/3798267248 wait complete. 2024-05-08T06:37:07.777 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T06:37:07.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.882+0000 7f0aa8714640 1 Processor -- start 2024-05-08T06:37:07.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.883+0000 7f0aa8714640 1 -- start start 2024-05-08T06:37:07.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.884+0000 7f0aa8714640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0aa01607d0 0x7f0aa00b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:07.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.884+0000 7f0aa8714640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0aa00d1270 0x7f0aa00d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:07.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.884+0000 7f0aa8714640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aa015d580 con 0x7f0aa01603f0 2024-05-08T06:37:07.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.884+0000 7f0aa8714640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa00d1c00 con 0x7f0aa01607d0 2024-05-08T06:37:07.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.884+0000 7f0aa8714640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa00b4e70 con 0x7f0aa00d1270 2024-05-08T06:37:07.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.884+0000 7f0aa5caf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0aa01607d0 0x7f0aa00b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:07.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.884+0000 7f0aa64b0640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0aa01603f0 0x7f0aa015ce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45884/0 (socket says 172.21.15.163:45884) 2024-05-08T06:37:07.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.884+0000 7f0aa64b0640 1 -- 172.21.15.163:0/2187012194 learned_addr learned my addr 172.21.15.163:0/2187012194 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:07.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.884+0000 7f0aa6cb1640 1 --2- 172.21.15.163:0/2187012194 >> v2:172.21.15.192:3300/0 conn(0x7f0aa00d1270 0x7f0aa00d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:07.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.884+0000 7f0aa5caf640 1 -- 172.21.15.163:0/2187012194 >> v2:172.21.15.192:3300/0 conn(0x7f0aa00d1270 msgr2=0x7f0aa00d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:07.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.884+0000 7f0aa5caf640 1 --2- 172.21.15.163:0/2187012194 >> v2:172.21.15.192:3300/0 conn(0x7f0aa00d1270 0x7f0aa00d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:07.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.884+0000 7f0aa5caf640 1 -- 172.21.15.163:0/2187012194 >> v1:172.21.15.163:6789/0 conn(0x7f0aa01603f0 legacy=0x7f0aa015ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:07.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.884+0000 7f0aa54ae640 1 -- 172.21.15.163:0/2187012194 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1975660135 0 0) 0x7f0aa015d580 con 0x7f0aa01603f0 2024-05-08T06:37:07.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.884+0000 7f0aa5caf640 1 -- 172.21.15.163:0/2187012194 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa00b5150 con 0x7f0aa01607d0 2024-05-08T06:37:07.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.885+0000 7f0aa5caf640 1 --2- 172.21.15.163:0/2187012194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aa01607d0 0x7f0aa00b5f10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00b42d0 tx=0x7f0a9002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3a657754cddaad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:07.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.885+0000 7f0aa54ae640 1 -- 172.21.15.163:0/2187012194 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90037070 con 0x7f0aa01607d0 2024-05-08T06:37:07.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.885+0000 7f0aa54ae640 1 -- 172.21.15.163:0/2187012194 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a90032440 con 0x7f0aa01607d0 2024-05-08T06:37:07.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.885+0000 7f0aa54ae640 1 -- 172.21.15.163:0/2187012194 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90032810 con 0x7f0aa01607d0 2024-05-08T06:37:07.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.885+0000 7f0aa8714640 1 -- 172.21.15.163:0/2187012194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aa01607d0 msgr2=0x7f0aa00b5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:07.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.885+0000 7f0aa8714640 1 --2- 172.21.15.163:0/2187012194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aa01607d0 0x7f0aa00b5f10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00b42d0 tx=0x7f0a9002a1a0 comp rx=0 tx=0).stop 2024-05-08T06:37:07.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.886+0000 7f0aa8714640 1 -- 172.21.15.163:0/2187012194 shutdown_connections 2024-05-08T06:37:07.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.886+0000 7f0aa8714640 1 --2- 172.21.15.163:0/2187012194 >> v2:172.21.15.192:3300/0 conn(0x7f0aa00d1270 0x7f0aa00d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:07.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.886+0000 7f0aa8714640 1 --2- 172.21.15.163:0/2187012194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aa01607d0 0x7f0aa00b5f10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:07.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.886+0000 7f0aa8714640 1 -- 172.21.15.163:0/2187012194 >> 172.21.15.163:0/2187012194 conn(0x7f0aa006c500 msgr2=0x7f0aa006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:07.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.886+0000 7f0aa8714640 1 -- 172.21.15.163:0/2187012194 shutdown_connections 2024-05-08T06:37:07.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.886+0000 7f0aa8714640 1 -- 172.21.15.163:0/2187012194 wait complete. 2024-05-08T06:37:07.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.887+0000 7f0aa8714640 1 Processor -- start 2024-05-08T06:37:07.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.887+0000 7f0aa8714640 1 -- start start 2024-05-08T06:37:07.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.887+0000 7f0aa8714640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0aa00d1270 0x7f0aa00b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:07.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.887+0000 7f0aa8714640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aa01603f0 0x7f0aa0158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:07.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa8714640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0aa01607d0 0x7f0aa006e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:07.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa8714640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa00b59f0 con 0x7f0aa01603f0 2024-05-08T06:37:07.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa8714640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa00b5710 con 0x7f0aa01607d0 2024-05-08T06:37:07.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa8714640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa0072840 con 0x7f0aa00d1270 2024-05-08T06:37:07.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa5caf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aa01603f0 0x7f0aa0158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:07.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa5caf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aa01603f0 0x7f0aa0158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41478/0 (socket says 172.21.15.163:41478) 2024-05-08T06:37:07.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa5caf640 1 -- 172.21.15.163:0/2057728670 learned_addr learned my addr 172.21.15.163:0/2057728670 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:07.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa64b0640 1 --2- 172.21.15.163:0/2057728670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0aa00d1270 0x7f0aa00b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:07.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa6cb1640 1 --2- 172.21.15.163:0/2057728670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0aa01607d0 0x7f0aa006e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:07.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa5caf640 1 -- 172.21.15.163:0/2057728670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0aa00d1270 msgr2=0x7f0aa00b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:07.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa5caf640 1 --2- 172.21.15.163:0/2057728670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0aa00d1270 0x7f0aa00b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:07.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa5caf640 1 -- 172.21.15.163:0/2057728670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0aa01607d0 msgr2=0x7f0aa006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:07.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa5caf640 1 --2- 172.21.15.163:0/2057728670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0aa01607d0 0x7f0aa006e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:07.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa5caf640 1 -- 172.21.15.163:0/2057728670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa00b1ec0 con 0x7f0aa01603f0 2024-05-08T06:37:07.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa5caf640 1 --2- 172.21.15.163:0/2057728670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aa01603f0 0x7f0aa0158050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0a90000c00 tx=0x7f0a900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:07.890 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0a96ffd640 1 -- 172.21.15.163:0/2057728670 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90037040 con 0x7f0aa01603f0 2024-05-08T06:37:07.890 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0a96ffd640 1 -- 172.21.15.163:0/2057728670 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a90042070 con 0x7f0aa01603f0 2024-05-08T06:37:07.890 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa8714640 1 -- 172.21.15.163:0/2057728670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa0156cd0 con 0x7f0aa01603f0 2024-05-08T06:37:07.890 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0a96ffd640 1 -- 172.21.15.163:0/2057728670 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90030bd0 con 0x7f0aa01603f0 2024-05-08T06:37:07.890 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.888+0000 7f0aa8714640 1 -- 172.21.15.163:0/2057728670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa0157290 con 0x7f0aa01603f0 2024-05-08T06:37:07.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.889+0000 7f0aa8714640 1 -- 172.21.15.163:0/2057728670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa006a550 con 0x7f0aa01603f0 2024-05-08T06:37:07.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.890+0000 7f0a96ffd640 1 -- 172.21.15.163:0/2057728670 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0a90047460 con 0x7f0aa01603f0 2024-05-08T06:37:07.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.890+0000 7f0a96ffd640 1 --2- 172.21.15.163:0/2057728670 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0a840762d0 0x7f0a84078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:07.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.890+0000 7f0a96ffd640 1 -- 172.21.15.163:0/2057728670 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0a900b8230 con 0x7f0aa01603f0 2024-05-08T06:37:07.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.890+0000 7f0aa64b0640 1 --2- 172.21.15.163:0/2057728670 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0a840762d0 0x7f0a84078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:07.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.891+0000 7f0aa64b0640 1 --2- 172.21.15.163:0/2057728670 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0a840762d0 0x7f0a84078790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c000f60 tx=0x7f0a9c007b10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:07.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:07.895+0000 7f0a96ffd640 1 -- 172.21.15.163:0/2057728670 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0a90032e20 con 0x7f0aa01603f0 2024-05-08T06:37:08.074 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.075+0000 7f0aa8714640 1 -- 172.21.15.163:0/2057728670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0aa0072e70 con 0x7f0aa01603f0 2024-05-08T06:37:08.077 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.078+0000 7f0a96ffd640 1 -- 172.21.15.163:0/2057728670 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f0a90081b90 con 0x7f0aa01603f0 2024-05-08T06:37:08.080 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.080+0000 7f0aa8714640 1 -- 172.21.15.163:0/2057728670 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0a840762d0 msgr2=0x7f0a84078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:08.080 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.080+0000 7f0aa8714640 1 --2- 172.21.15.163:0/2057728670 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0a840762d0 0x7f0a84078790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c000f60 tx=0x7f0a9c007b10 comp rx=0 tx=0).stop 2024-05-08T06:37:08.080 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.080+0000 7f0aa8714640 1 -- 172.21.15.163:0/2057728670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aa01603f0 msgr2=0x7f0aa0158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:08.080 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.080+0000 7f0aa8714640 1 --2- 172.21.15.163:0/2057728670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aa01603f0 0x7f0aa0158050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0a90000c00 tx=0x7f0a900046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.080 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.081+0000 7f0aa8714640 1 -- 172.21.15.163:0/2057728670 shutdown_connections 2024-05-08T06:37:08.080 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.081+0000 7f0aa8714640 1 --2- 172.21.15.163:0/2057728670 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0a840762d0 0x7f0a84078790 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.080 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.081+0000 7f0aa8714640 1 --2- 172.21.15.163:0/2057728670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0aa01607d0 0x7f0aa006e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.080 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.081+0000 7f0aa8714640 1 --2- 172.21.15.163:0/2057728670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aa01603f0 0x7f0aa0158050 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.081 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.081+0000 7f0aa8714640 1 --2- 172.21.15.163:0/2057728670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0aa00d1270 0x7f0aa00b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.081 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.081+0000 7f0aa8714640 1 -- 172.21.15.163:0/2057728670 >> 172.21.15.163:0/2057728670 conn(0x7f0aa006c500 msgr2=0x7f0aa00aea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:08.081 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.081+0000 7f0aa8714640 1 -- 172.21.15.163:0/2057728670 shutdown_connections 2024-05-08T06:37:08.081 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.081+0000 7f0aa8714640 1 -- 172.21.15.163:0/2057728670 wait complete. 2024-05-08T06:37:08.091 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T06:37:08.091 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T06:37:08.131 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T06:37:08.174 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T06:37:08.174 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:37:08.174 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:08.190 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:37:08.190 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:08.247 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:37:08.247 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:08.303 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T06:37:08.444 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.445+0000 7fc505e28640 1 Processor -- start 2024-05-08T06:37:08.444 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.445+0000 7fc505e28640 1 -- start start 2024-05-08T06:37:08.445 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.446+0000 7fc505e28640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc5000b4a20 0x7fc500072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:08.446 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.446+0000 7fc505e28640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc50015fa50 0x7fc50006e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:08.446 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.446+0000 7fc505e28640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5000745f0 con 0x7fc5000b4e00 2024-05-08T06:37:08.446 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.446+0000 7fc505e28640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc500073320 con 0x7fc5000b4a20 2024-05-08T06:37:08.446 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.446+0000 7fc505e28640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc500074160 con 0x7fc50015fa50 2024-05-08T06:37:08.446 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.446+0000 7fc4fffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc50015fa50 0x7fc50006e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:08.446 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.446+0000 7fc4ff7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc5000b4a20 0x7fc500072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:08.446 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.446+0000 7fc4fffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc50015fa50 0x7fc50006e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:36298/0 (socket says 172.21.15.163:36298) 2024-05-08T06:37:08.446 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.446+0000 7fc4fffff640 1 -- 172.21.15.163:0/3220504827 learned_addr learned my addr 172.21.15.163:0/3220504827 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:08.446 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.446+0000 7fc4fffff640 1 -- 172.21.15.163:0/3220504827 >> v2:172.21.15.192:3300/0 conn(0x7fc5000b4a20 msgr2=0x7fc500072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:08.446 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.446+0000 7fc4fffff640 1 --2- 172.21.15.163:0/3220504827 >> v2:172.21.15.192:3300/0 conn(0x7fc5000b4a20 0x7fc500072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.446 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.446+0000 7fc4fffff640 1 -- 172.21.15.163:0/3220504827 >> v1:172.21.15.192:6789/0 conn(0x7fc5000b4e00 legacy=0x7fc50015f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:08.446 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.446+0000 7fc4fffff640 1 -- 172.21.15.163:0/3220504827 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc500073550 con 0x7fc50015fa50 2024-05-08T06:37:08.446 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.446+0000 7fc4ff7fe640 1 --2- 172.21.15.163:0/3220504827 >> v2:172.21.15.192:3300/0 conn(0x7fc5000b4a20 0x7fc500072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:08.448 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.447+0000 7fc4fffff640 1 --2- 172.21.15.163:0/3220504827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc50015fa50 0x7fc50006e380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009e50 tx=0x7fc4f802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da3993968b2e1d56 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:08.448 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.447+0000 7fc4feffd640 1 -- 172.21.15.163:0/3220504827 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8036070 con 0x7fc50015fa50 2024-05-08T06:37:08.448 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.447+0000 7fc4feffd640 1 -- 172.21.15.163:0/3220504827 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f8031600 con 0x7fc50015fa50 2024-05-08T06:37:08.448 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.447+0000 7fc4feffd640 1 -- 172.21.15.163:0/3220504827 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f80319d0 con 0x7fc50015fa50 2024-05-08T06:37:08.448 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.448+0000 7fc505e28640 1 -- 172.21.15.163:0/3220504827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc50015fa50 msgr2=0x7fc50006e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:08.448 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.448+0000 7fc505e28640 1 --2- 172.21.15.163:0/3220504827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc50015fa50 0x7fc50006e380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009e50 tx=0x7fc4f802a710 comp rx=0 tx=0).stop 2024-05-08T06:37:08.448 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.448+0000 7fc505e28640 1 -- 172.21.15.163:0/3220504827 shutdown_connections 2024-05-08T06:37:08.448 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.448+0000 7fc505e28640 1 --2- 172.21.15.163:0/3220504827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc50015fa50 0x7fc50006e380 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.448+0000 7fc505e28640 1 --2- 172.21.15.163:0/3220504827 >> v2:172.21.15.192:3300/0 conn(0x7fc5000b4a20 0x7fc500072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.448+0000 7fc505e28640 1 -- 172.21.15.163:0/3220504827 >> 172.21.15.163:0/3220504827 conn(0x7fc50006c740 msgr2=0x7fc50006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:08.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.448+0000 7fc505e28640 1 -- 172.21.15.163:0/3220504827 shutdown_connections 2024-05-08T06:37:08.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.448+0000 7fc505e28640 1 -- 172.21.15.163:0/3220504827 wait complete. 2024-05-08T06:37:08.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.449+0000 7fc505e28640 1 Processor -- start 2024-05-08T06:37:08.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.449+0000 7fc505e28640 1 -- start start 2024-05-08T06:37:08.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc505e28640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5000b4a20 0x7fc5001469e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:08.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc505e28640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5000b4e00 0x7fc5001f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:08.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc505e28640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50015fa50 0x7fc5000cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:08.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc505e28640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc500073df0 con 0x7fc5000b4e00 2024-05-08T06:37:08.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc505e28640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc500073b10 con 0x7fc50015fa50 2024-05-08T06:37:08.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc505e28640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc5000b5290 con 0x7fc5000b4a20 2024-05-08T06:37:08.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc4f7fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5000b4e00 0x7fc5001f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:08.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc4f7fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5000b4e00 0x7fc5001f18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41490/0 (socket says 172.21.15.163:41490) 2024-05-08T06:37:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc4f7fff640 1 -- 172.21.15.163:0/3988994331 learned_addr learned my addr 172.21.15.163:0/3988994331 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc4ff7fe640 1 --2- 172.21.15.163:0/3988994331 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5000b4a20 0x7fc5001469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc4fffff640 1 --2- 172.21.15.163:0/3988994331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50015fa50 0x7fc5000cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc4f7fff640 1 -- 172.21.15.163:0/3988994331 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5000b4a20 msgr2=0x7fc5001469e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc4f7fff640 1 --2- 172.21.15.163:0/3988994331 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5000b4a20 0x7fc5001469e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc4f7fff640 1 -- 172.21.15.163:0/3988994331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50015fa50 msgr2=0x7fc5000cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc4f7fff640 1 --2- 172.21.15.163:0/3988994331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50015fa50 0x7fc5000cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.450+0000 7fc4f7fff640 1 -- 172.21.15.163:0/3988994331 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5000cf730 con 0x7fc5000b4e00 2024-05-08T06:37:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.451+0000 7fc4f7fff640 1 --2- 172.21.15.163:0/3988994331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5000b4e00 0x7fc5001f18d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800c480 tx=0x7fc4e800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.451+0000 7fc4fcff9640 1 -- 172.21.15.163:0/3988994331 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80052d0 con 0x7fc5000b4e00 2024-05-08T06:37:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.451+0000 7fc4fcff9640 1 -- 172.21.15.163:0/3988994331 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e8005470 con 0x7fc5000b4e00 2024-05-08T06:37:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.451+0000 7fc505e28640 1 -- 172.21.15.163:0/3988994331 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5000cf9d0 con 0x7fc5000b4e00 2024-05-08T06:37:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.451+0000 7fc4fcff9640 1 -- 172.21.15.163:0/3988994331 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8012870 con 0x7fc5000b4e00 2024-05-08T06:37:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.451+0000 7fc505e28640 1 -- 172.21.15.163:0/3988994331 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5000cff90 con 0x7fc5000b4e00 2024-05-08T06:37:08.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.452+0000 7fc505e28640 1 -- 172.21.15.163:0/3988994331 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc50015c8b0 con 0x7fc5000b4e00 2024-05-08T06:37:08.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.453+0000 7fc4fcff9640 1 -- 172.21.15.163:0/3988994331 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc4e8012a10 con 0x7fc5000b4e00 2024-05-08T06:37:08.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.453+0000 7fc4fcff9640 1 --2- 172.21.15.163:0/3988994331 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc4e0076300 0x7fc4e00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:08.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.453+0000 7fc4fcff9640 1 -- 172.21.15.163:0/3988994331 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc4e8097060 con 0x7fc5000b4e00 2024-05-08T06:37:08.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.453+0000 7fc4ff7fe640 1 --2- 172.21.15.163:0/3988994331 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc4e0076300 0x7fc4e00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:08.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.453+0000 7fc4ff7fe640 1 --2- 172.21.15.163:0/3988994331 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc4e0076300 0x7fc4e00787c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009560 tx=0x7fc4f0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:08.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.457+0000 7fc4fcff9640 1 -- 172.21.15.163:0/3988994331 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc50015c8b0 con 0x7fc5000b4e00 2024-05-08T06:37:08.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.652+0000 7fc505e28640 1 -- 172.21.15.163:0/3988994331 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc5000728f0 con 0x7fc5000b4e00 2024-05-08T06:37:08.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.652+0000 7fc4fcff9640 1 -- 172.21.15.163:0/3988994331 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fc4e8060400 con 0x7fc5000b4e00 2024-05-08T06:37:08.651 INFO:teuthology.orchestra.run.smithi163.stderr:listed 0 entries 2024-05-08T06:37:08.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.654+0000 7fc505e28640 1 -- 172.21.15.163:0/3988994331 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc4e0076300 msgr2=0x7fc4e00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:08.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.654+0000 7fc505e28640 1 --2- 172.21.15.163:0/3988994331 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc4e0076300 0x7fc4e00787c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009560 tx=0x7fc4f0009190 comp rx=0 tx=0).stop 2024-05-08T06:37:08.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.654+0000 7fc505e28640 1 -- 172.21.15.163:0/3988994331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5000b4e00 msgr2=0x7fc5001f18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:08.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.654+0000 7fc505e28640 1 --2- 172.21.15.163:0/3988994331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5000b4e00 0x7fc5001f18d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800c480 tx=0x7fc4e800c940 comp rx=0 tx=0).stop 2024-05-08T06:37:08.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.654+0000 7fc505e28640 1 -- 172.21.15.163:0/3988994331 shutdown_connections 2024-05-08T06:37:08.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.654+0000 7fc505e28640 1 --2- 172.21.15.163:0/3988994331 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc4e0076300 0x7fc4e00787c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.654+0000 7fc505e28640 1 --2- 172.21.15.163:0/3988994331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50015fa50 0x7fc5000cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.654+0000 7fc505e28640 1 --2- 172.21.15.163:0/3988994331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5000b4e00 0x7fc5001f18d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.654+0000 7fc505e28640 1 --2- 172.21.15.163:0/3988994331 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5000b4a20 0x7fc5001469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.654+0000 7fc505e28640 1 -- 172.21.15.163:0/3988994331 >> 172.21.15.163:0/3988994331 conn(0x7fc50006c740 msgr2=0x7fc50015e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:08.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.654+0000 7fc505e28640 1 -- 172.21.15.163:0/3988994331 shutdown_connections 2024-05-08T06:37:08.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.654+0000 7fc505e28640 1 -- 172.21.15.163:0/3988994331 wait complete. 2024-05-08T06:37:08.664 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T06:37:08.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:08.665 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-08T06:37:08.691 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:37:08.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:08.692 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T06:37:08.758 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:37:08.758 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:08.777 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:37:08.778 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:08.836 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:37:08.836 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:08.894 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T06:37:08.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.997+0000 7f083032b640 1 Processor -- start 2024-05-08T06:37:08.996 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.997+0000 7f083032b640 1 -- start start 2024-05-08T06:37:08.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.998+0000 7f083032b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f082815f500 0x7f082806f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:08.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.998+0000 7f083032b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f08280b3f60 0x7f08280b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:08.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.998+0000 7f083032b640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08280b0fc0 con 0x7f08280b7140 2024-05-08T06:37:08.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.998+0000 7f083032b640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f08280b18f0 con 0x7f08280b3f60 2024-05-08T06:37:08.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.998+0000 7f083032b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f08280b4910 con 0x7f082815f500 2024-05-08T06:37:08.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.998+0000 7f082e8c8640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f08280b3f60 0x7f08280b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:08.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.998+0000 7f082d8c6640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f08280b7140 0x7f0828073b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45892/0 (socket says 172.21.15.163:45892) 2024-05-08T06:37:08.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.998+0000 7f082d8c6640 1 -- 172.21.15.163:0/2175454885 learned_addr learned my addr 172.21.15.163:0/2175454885 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:08.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.998+0000 7f082e0c7640 1 --2- 172.21.15.163:0/2175454885 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f082815f500 0x7f082806f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:08.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.998+0000 7f082e8c8640 1 -- 172.21.15.163:0/2175454885 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f082815f500 msgr2=0x7f082806f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:08.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.998+0000 7f082e8c8640 1 --2- 172.21.15.163:0/2175454885 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f082815f500 0x7f082806f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:08.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.998+0000 7f082e8c8640 1 -- 172.21.15.163:0/2175454885 >> v1:172.21.15.163:6789/0 conn(0x7f08280b7140 legacy=0x7f0828073b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:08.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.998+0000 7f082e8c8640 1 -- 172.21.15.163:0/2175454885 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08280b12a0 con 0x7f08280b3f60 2024-05-08T06:37:08.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.998+0000 7f082d0c5640 1 -- 172.21.15.163:0/2175454885 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3740047262 0 0) 0x7f08280b0fc0 con 0x7f08280b7140 2024-05-08T06:37:08.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.999+0000 7f082e8c8640 1 --2- 172.21.15.163:0/2175454885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08280b3f60 0x7f08280b43d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0820009d90 tx=0x7f082002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65cf84de9f30c82 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:08.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.999+0000 7f082d0c5640 1 -- 172.21.15.163:0/2175454885 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820007740 con 0x7f08280b3f60 2024-05-08T06:37:08.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.999+0000 7f082d0c5640 1 -- 172.21.15.163:0/2175454885 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08200078e0 con 0x7f08280b3f60 2024-05-08T06:37:08.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:08.999+0000 7f082d0c5640 1 -- 172.21.15.163:0/2175454885 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820007cb0 con 0x7f08280b3f60 2024-05-08T06:37:09.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.000+0000 7f083032b640 1 -- 172.21.15.163:0/2175454885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08280b3f60 msgr2=0x7f08280b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:09.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.000+0000 7f083032b640 1 --2- 172.21.15.163:0/2175454885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08280b3f60 0x7f08280b43d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0820009d90 tx=0x7f082002a660 comp rx=0 tx=0).stop 2024-05-08T06:37:09.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.000+0000 7f083032b640 1 -- 172.21.15.163:0/2175454885 shutdown_connections 2024-05-08T06:37:09.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.000+0000 7f083032b640 1 --2- 172.21.15.163:0/2175454885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08280b3f60 0x7f08280b43d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.000+0000 7f083032b640 1 --2- 172.21.15.163:0/2175454885 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f082815f500 0x7f082806f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.000+0000 7f083032b640 1 -- 172.21.15.163:0/2175454885 >> 172.21.15.163:0/2175454885 conn(0x7f082806d610 msgr2=0x7f082806d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:09.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.000+0000 7f083032b640 1 -- 172.21.15.163:0/2175454885 shutdown_connections 2024-05-08T06:37:09.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.000+0000 7f083032b640 1 -- 172.21.15.163:0/2175454885 wait complete. 2024-05-08T06:37:09.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.001+0000 7f083032b640 1 Processor -- start 2024-05-08T06:37:09.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.001+0000 7f083032b640 1 -- start start 2024-05-08T06:37:09.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f083032b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08280b3f60 0x7f082814c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:09.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f083032b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08280b7140 0x7f0828152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:09.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f083032b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f082815f500 0x7f0828152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:09.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f083032b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f082815fc70 con 0x7f08280b3f60 2024-05-08T06:37:09.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f083032b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f082806fa10 con 0x7f08280b7140 2024-05-08T06:37:09.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f083032b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f082815fdf0 con 0x7f082815f500 2024-05-08T06:37:09.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f082e0c7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08280b3f60 0x7f082814c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:09.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f082e0c7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08280b3f60 0x7f082814c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41504/0 (socket says 172.21.15.163:41504) 2024-05-08T06:37:09.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f082e0c7640 1 -- 172.21.15.163:0/3843938428 learned_addr learned my addr 172.21.15.163:0/3843938428 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:09.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f082d8c6640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08280b7140 0x7f0828152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:09.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f082e0c7640 1 -- 172.21.15.163:0/3843938428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f082815f500 msgr2=0x7f0828152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:09.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f082e8c8640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f082815f500 0x7f0828152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:09.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f082e0c7640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f082815f500 0x7f0828152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f082e0c7640 1 -- 172.21.15.163:0/3843938428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08280b7140 msgr2=0x7f0828152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:09.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f082e0c7640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08280b7140 0x7f0828152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f082e0c7640 1 -- 172.21.15.163:0/3843938428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08281f5ce0 con 0x7f08280b3f60 2024-05-08T06:37:09.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f082d8c6640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08280b7140 0x7f0828152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:09.003 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f082e0c7640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08280b3f60 0x7f082814c3d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f082400d5f0 tx=0x7f082400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:09.004 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f081effd640 1 -- 172.21.15.163:0/3843938428 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08240125f0 con 0x7f08280b3f60 2024-05-08T06:37:09.004 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f081effd640 1 -- 172.21.15.163:0/3843938428 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0824019070 con 0x7f08280b3f60 2024-05-08T06:37:09.004 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f083032b640 1 -- 172.21.15.163:0/3843938428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08281f4f20 con 0x7f08280b3f60 2024-05-08T06:37:09.004 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f081effd640 1 -- 172.21.15.163:0/3843938428 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08240129d0 con 0x7f08280b3f60 2024-05-08T06:37:09.004 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.002+0000 7f083032b640 1 -- 172.21.15.163:0/3843938428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08281f57c0 con 0x7f08280b3f60 2024-05-08T06:37:09.004 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.003+0000 7f083032b640 1 -- 172.21.15.163:0/3843938428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08281f5310 con 0x7f08280b3f60 2024-05-08T06:37:09.004 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.004+0000 7f081effd640 1 -- 172.21.15.163:0/3843938428 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0824012b70 con 0x7f08280b3f60 2024-05-08T06:37:09.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.004+0000 7f081effd640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f080c0762d0 0x7f080c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:09.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.005+0000 7f081effd640 1 -- 172.21.15.163:0/3843938428 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0824097a30 con 0x7f08280b3f60 2024-05-08T06:37:09.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.005+0000 7f082d8c6640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f080c0762d0 0x7f080c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:09.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.005+0000 7f082d8c6640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f080c0762d0 0x7f080c078790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0828162010 tx=0x7f0818009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:09.008 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.009+0000 7f081effd640 1 -- 172.21.15.163:0/3843938428 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0824061390 con 0x7f08280b3f60 2024-05-08T06:37:09.202 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.202+0000 7f083032b640 1 -- 172.21.15.163:0/3843938428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f08281f50f0 con 0x7f08280b3f60 2024-05-08T06:37:09.202 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.203+0000 7f081effd640 1 -- 172.21.15.163:0/3843938428 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f0824060c90 con 0x7f08280b3f60 2024-05-08T06:37:09.202 INFO:teuthology.orchestra.run.smithi163.stderr:listed 0 entries 2024-05-08T06:37:09.204 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.205+0000 7f083032b640 1 -- 172.21.15.163:0/3843938428 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f080c0762d0 msgr2=0x7f080c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:09.204 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.205+0000 7f083032b640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f080c0762d0 0x7f080c078790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0828162010 tx=0x7f0818009040 comp rx=0 tx=0).stop 2024-05-08T06:37:09.204 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.205+0000 7f083032b640 1 -- 172.21.15.163:0/3843938428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08280b3f60 msgr2=0x7f082814c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:09.204 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.205+0000 7f083032b640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08280b3f60 0x7f082814c3d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f082400d5f0 tx=0x7f082400dab0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.205 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.205+0000 7f083032b640 1 -- 172.21.15.163:0/3843938428 shutdown_connections 2024-05-08T06:37:09.205 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.205+0000 7f083032b640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f080c0762d0 0x7f080c078790 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.205 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.205+0000 7f083032b640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f082815f500 0x7f0828152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.205 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.205+0000 7f083032b640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08280b7140 0x7f0828152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.205 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.205+0000 7f083032b640 1 --2- 172.21.15.163:0/3843938428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08280b3f60 0x7f082814c3d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.205 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.205+0000 7f083032b640 1 -- 172.21.15.163:0/3843938428 >> 172.21.15.163:0/3843938428 conn(0x7f082806d610 msgr2=0x7f082815ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:09.205 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.205+0000 7f083032b640 1 -- 172.21.15.163:0/3843938428 shutdown_connections 2024-05-08T06:37:09.205 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.205+0000 7f083032b640 1 -- 172.21.15.163:0/3843938428 wait complete. 2024-05-08T06:37:09.215 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-08T06:37:09.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:09.215 DEBUG:teuthology.orchestra.run.smithi192:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-08T06:37:09.248 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T06:37:09.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:09.248 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T06:37:09.320 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:37:09.320 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:09.336 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:37:09.336 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:09.393 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:37:09.393 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:09.449 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T06:37:09.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.590+0000 7fc0ec16a640 1 Processor -- start 2024-05-08T06:37:09.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.590+0000 7fc0ec16a640 1 -- start start 2024-05-08T06:37:09.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.591+0000 7fc0ec16a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc0e40d3d10 0x7fc0e41629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:09.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.591+0000 7fc0ec16a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e4162f00 0x7fc0e406d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:09.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.591+0000 7fc0ec16a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0e40d1560 con 0x7fc0e40d3930 2024-05-08T06:37:09.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.591+0000 7fc0ec16a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e4163390 con 0x7fc0e40d3d10 2024-05-08T06:37:09.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.591+0000 7fc0ec16a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e40d0cc0 con 0x7fc0e4162f00 2024-05-08T06:37:09.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.591+0000 7fc0e9f06640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc0e40d3930 0x7fc0e40d59a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60538/0 (socket says 172.21.15.163:60538) 2024-05-08T06:37:09.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.591+0000 7fc0e9f06640 1 -- 172.21.15.163:0/673033730 learned_addr learned my addr 172.21.15.163:0/673033730 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:09.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.591+0000 7fc0e8f04640 1 -- 172.21.15.163:0/673033730 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3569066062 0 0) 0x7fc0e40d1560 con 0x7fc0e40d3930 2024-05-08T06:37:09.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.591+0000 7fc0e8f04640 1 -- 172.21.15.163:0/673033730 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0d0003520 con 0x7fc0e40d3930 2024-05-08T06:37:09.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.591+0000 7fc0ea707640 1 --2- 172.21.15.163:0/673033730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e4162f00 0x7fc0e406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:09.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.591+0000 7fc0e9705640 1 --2- 172.21.15.163:0/673033730 >> v2:172.21.15.192:3300/0 conn(0x7fc0e40d3d10 0x7fc0e41629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:09.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e8f04640 1 -- 172.21.15.163:0/673033730 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3016480412 0 0) 0x7fc0d0003520 con 0x7fc0e40d3930 2024-05-08T06:37:09.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e8f04640 1 -- 172.21.15.163:0/673033730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e4162f00 msgr2=0x7fc0e406d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:09.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e8f04640 1 --2- 172.21.15.163:0/673033730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e4162f00 0x7fc0e406d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e8f04640 1 -- 172.21.15.163:0/673033730 >> v2:172.21.15.192:3300/0 conn(0x7fc0e40d3d10 msgr2=0x7fc0e41629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:09.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e8f04640 1 --2- 172.21.15.163:0/673033730 >> v2:172.21.15.192:3300/0 conn(0x7fc0e40d3d10 0x7fc0e41629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e8f04640 1 -- 172.21.15.163:0/673033730 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e40d0fa0 con 0x7fc0e40d3930 2024-05-08T06:37:09.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e8f04640 1 -- 172.21.15.163:0/673033730 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7fc0e00016f0 con 0x7fc0e40d3930 2024-05-08T06:37:09.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e8f04640 1 -- 172.21.15.163:0/673033730 >> v1:172.21.15.163:6789/0 conn(0x7fc0e40d3930 legacy=0x7fc0e40d59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e8f04640 1 --2- 172.21.15.163:0/673033730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0d0003a00 0x7fc0d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e8f04640 1 --2- 172.21.15.163:0/673033730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e4162f00 0x7fc0d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e8f04640 1 --2- 172.21.15.163:0/673033730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0d0004d40 0x7fc0d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e8f04640 1 -- 172.21.15.163:0/673033730 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e40d0cc0 con 0x7fc0d0004d40 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e8f04640 1 -- 172.21.15.163:0/673033730 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d000b9c0 con 0x7fc0d0003a00 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e8f04640 1 -- 172.21.15.163:0/673033730 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d000bb70 con 0x7fc0e4162f00 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0ea707640 1 --2- 172.21.15.163:0/673033730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0d0004d40 0x7fc0d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e9705640 1 --2- 172.21.15.163:0/673033730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e4162f00 0x7fc0d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0e9f06640 1 --2- 172.21.15.163:0/673033730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0d0003a00 0x7fc0d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0ea707640 1 -- 172.21.15.163:0/673033730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e4162f00 msgr2=0x7fc0d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0ea707640 1 --2- 172.21.15.163:0/673033730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e4162f00 0x7fc0d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0ea707640 1 -- 172.21.15.163:0/673033730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0d0003a00 msgr2=0x7fc0d0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0ea707640 1 --2- 172.21.15.163:0/673033730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0d0003a00 0x7fc0d0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.592+0000 7fc0ea707640 1 -- 172.21.15.163:0/673033730 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d0003520 con 0x7fc0d0004d40 2024-05-08T06:37:09.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.593+0000 7fc0ea707640 1 --2- 172.21.15.163:0/673033730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0d0004d40 0x7fc0d00051c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009fc0 tx=0x7fc0d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53dc93e41d71c886 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:09.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.593+0000 7fc0e8f04640 1 -- 172.21.15.163:0/673033730 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc009c10 con 0x7fc0d0004d40 2024-05-08T06:37:09.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.593+0000 7fc0e8f04640 1 -- 172.21.15.163:0/673033730 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0dc036070 con 0x7fc0d0004d40 2024-05-08T06:37:09.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.593+0000 7fc0e8f04640 1 -- 172.21.15.163:0/673033730 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc031880 con 0x7fc0d0004d40 2024-05-08T06:37:09.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.593+0000 7fc0ec16a640 1 -- 172.21.15.163:0/673033730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0d0004d40 msgr2=0x7fc0d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:09.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.593+0000 7fc0ec16a640 1 --2- 172.21.15.163:0/673033730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0d0004d40 0x7fc0d00051c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009fc0 tx=0x7fc0d0005830 comp rx=0 tx=0).stop 2024-05-08T06:37:09.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.593+0000 7fc0ea707640 1 -- 172.21.15.163:0/673033730 reap_dead start 2024-05-08T06:37:09.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.594+0000 7fc0ec16a640 1 -- 172.21.15.163:0/673033730 shutdown_connections 2024-05-08T06:37:09.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.594+0000 7fc0ec16a640 1 -- 172.21.15.163:0/673033730 >> 172.21.15.163:0/673033730 conn(0x7fc0e408cfa0 msgr2=0x7fc0e40d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:09.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.594+0000 7fc0ec16a640 1 -- 172.21.15.163:0/673033730 shutdown_connections 2024-05-08T06:37:09.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.594+0000 7fc0ec16a640 1 -- 172.21.15.163:0/673033730 wait complete. 2024-05-08T06:37:09.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.595+0000 7fc0ec16a640 1 Processor -- start 2024-05-08T06:37:09.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.595+0000 7fc0ec16a640 1 -- start start 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.595+0000 7fc0ec16a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0d0004d40 0x7fc0e4159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ec16a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e40cb900 0x7fc0e41f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ec16a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0e415c940 0x7fc0e415a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ec16a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e40d0970 con 0x7fc0e415c940 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ec16a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e40d60b0 con 0x7fc0e40cb900 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ec16a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e406d960 con 0x7fc0d0004d40 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ea707640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0e415c940 0x7fc0e415a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ea707640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0e415c940 0x7fc0e415a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40440/0 (socket says 172.21.15.163:40440) 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ea707640 1 -- 172.21.15.163:0/1774466804 learned_addr learned my addr 172.21.15.163:0/1774466804 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0e9f06640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0d0004d40 0x7fc0e4159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0e9705640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e40cb900 0x7fc0e41f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ea707640 1 -- 172.21.15.163:0/1774466804 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0d0004d40 msgr2=0x7fc0e4159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ea707640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0d0004d40 0x7fc0e4159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ea707640 1 -- 172.21.15.163:0/1774466804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e40cb900 msgr2=0x7fc0e41f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ea707640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e40cb900 0x7fc0e41f4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ea707640 1 -- 172.21.15.163:0/1774466804 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e4159950 con 0x7fc0e415c940 2024-05-08T06:37:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0e9705640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e40cb900 0x7fc0e41f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ea707640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0e415c940 0x7fc0e415a570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc033040 tx=0x7fc0dc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0da7fc640 1 -- 172.21.15.163:0/1774466804 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc009c10 con 0x7fc0e415c940 2024-05-08T06:37:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0da7fc640 1 -- 172.21.15.163:0/1774466804 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0dc031dd0 con 0x7fc0e415c940 2024-05-08T06:37:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ec16a640 1 -- 172.21.15.163:0/1774466804 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e40c8e60 con 0x7fc0e415c940 2024-05-08T06:37:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0da7fc640 1 -- 172.21.15.163:0/1774466804 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc003bd0 con 0x7fc0e415c940 2024-05-08T06:37:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.596+0000 7fc0ec16a640 1 -- 172.21.15.163:0/1774466804 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e40c9420 con 0x7fc0e415c940 2024-05-08T06:37:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.597+0000 7fc0ec16a640 1 -- 172.21.15.163:0/1774466804 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e406a4b0 con 0x7fc0e415c940 2024-05-08T06:37:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.598+0000 7fc0da7fc640 1 -- 172.21.15.163:0/1774466804 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc0dc005d10 con 0x7fc0e415c940 2024-05-08T06:37:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.598+0000 7fc0da7fc640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc0c80762d0 0x7fc0c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.598+0000 7fc0da7fc640 1 -- 172.21.15.163:0/1774466804 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc0dc0b8ce0 con 0x7fc0e415c940 2024-05-08T06:37:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.599+0000 7fc0e9f06640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc0c80762d0 0x7fc0c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:09.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.599+0000 7fc0e9f06640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc0c80762d0 0x7fc0c8078790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0007cb0 tx=0x7fc0e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:09.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.603+0000 7fc0da7fc640 1 -- 172.21.15.163:0/1774466804 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc0dc04a060 con 0x7fc0e415c940 2024-05-08T06:37:09.799 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.799+0000 7fc0ec16a640 1 -- 172.21.15.163:0/1774466804 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc0e40b4900 con 0x7fc0e415c940 2024-05-08T06:37:09.799 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.799+0000 7fc0da7fc640 1 -- 172.21.15.163:0/1774466804 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fc0dc0826d0 con 0x7fc0e415c940 2024-05-08T06:37:09.799 INFO:teuthology.orchestra.run.smithi163.stderr:listed 0 entries 2024-05-08T06:37:09.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.801+0000 7fc0ec16a640 1 -- 172.21.15.163:0/1774466804 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc0c80762d0 msgr2=0x7fc0c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:09.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.801+0000 7fc0ec16a640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc0c80762d0 0x7fc0c8078790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0007cb0 tx=0x7fc0e0009040 comp rx=0 tx=0).stop 2024-05-08T06:37:09.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.801+0000 7fc0ec16a640 1 -- 172.21.15.163:0/1774466804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0e415c940 msgr2=0x7fc0e415a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:09.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.801+0000 7fc0ec16a640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0e415c940 0x7fc0e415a570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc033040 tx=0x7fc0dc009750 comp rx=0 tx=0).stop 2024-05-08T06:37:09.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.801+0000 7fc0ec16a640 1 -- 172.21.15.163:0/1774466804 shutdown_connections 2024-05-08T06:37:09.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.801+0000 7fc0ec16a640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc0c80762d0 0x7fc0c8078790 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.801+0000 7fc0ec16a640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0e415c940 0x7fc0e415a570 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.801+0000 7fc0ec16a640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e40cb900 0x7fc0e41f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.801+0000 7fc0ec16a640 1 --2- 172.21.15.163:0/1774466804 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0d0004d40 0x7fc0e4159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:09.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.801+0000 7fc0ec16a640 1 -- 172.21.15.163:0/1774466804 >> 172.21.15.163:0/1774466804 conn(0x7fc0e408cfa0 msgr2=0x7fc0e40b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:09.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.801+0000 7fc0ec16a640 1 -- 172.21.15.163:0/1774466804 shutdown_connections 2024-05-08T06:37:09.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:09.801+0000 7fc0ec16a640 1 -- 172.21.15.163:0/1774466804 wait complete. 2024-05-08T06:37:09.813 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-08T06:37:09.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:09.813 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-08T06:37:09.839 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:37:09.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:09.839 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T06:37:09.895 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:37:09.895 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:09.914 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:37:09.914 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:09.974 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:37:09.974 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:10.033 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T06:37:10.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.134+0000 7f38e9fb6640 1 Processor -- start 2024-05-08T06:37:10.134 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.134+0000 7f38e9fb6640 1 -- start start 2024-05-08T06:37:10.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.135+0000 7f38e9fb6640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f38e413ff40 0x7f38e4071d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.135+0000 7f38e9fb6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e40722b0 0x7f38e41427e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.135+0000 7f38e9fb6640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38e40727c0 con 0x7f38e4142d20 2024-05-08T06:37:10.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.135+0000 7f38e9fb6640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f38e4072d80 con 0x7f38e413ff40 2024-05-08T06:37:10.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.135+0000 7f38e9fb6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f38e4072aa0 con 0x7f38e40722b0 2024-05-08T06:37:10.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.135+0000 7f38e37fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f38e413ff40 0x7f38e4071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.135+0000 7f38e37fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f38e413ff40 0x7f38e4071d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40442/0 (socket says 172.21.15.163:40442) 2024-05-08T06:37:10.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.135+0000 7f38e37fe640 1 -- 172.21.15.163:0/3072653653 learned_addr learned my addr 172.21.15.163:0/3072653653 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:10.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.135+0000 7f38e37fe640 1 -- 172.21.15.163:0/3072653653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e40722b0 msgr2=0x7f38e41427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:10.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.135+0000 7f38e37fe640 1 --2- 172.21.15.163:0/3072653653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e40722b0 0x7f38e41427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.135+0000 7f38e37fe640 1 -- 172.21.15.163:0/3072653653 >> v1:172.21.15.192:6789/0 conn(0x7f38e4142d20 legacy=0x7f38e4160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:10.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.135+0000 7f38e37fe640 1 -- 172.21.15.163:0/3072653653 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e41407f0 con 0x7f38e413ff40 2024-05-08T06:37:10.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.136+0000 7f38e37fe640 1 --2- 172.21.15.163:0/3072653653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e413ff40 0x7f38e4071d70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f38d40098a0 tx=0x7f38d402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8587d5779b822f8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:10.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.136+0000 7f38e27fc640 1 -- 172.21.15.163:0/3072653653 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4037070 con 0x7f38e413ff40 2024-05-08T06:37:10.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.136+0000 7f38e27fc640 1 -- 172.21.15.163:0/3072653653 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38d4032440 con 0x7f38e413ff40 2024-05-08T06:37:10.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.136+0000 7f38e27fc640 1 -- 172.21.15.163:0/3072653653 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4032810 con 0x7f38e413ff40 2024-05-08T06:37:10.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.137+0000 7f38e9fb6640 1 -- 172.21.15.163:0/3072653653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e413ff40 msgr2=0x7f38e4071d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.137+0000 7f38e9fb6640 1 --2- 172.21.15.163:0/3072653653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e413ff40 0x7f38e4071d70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f38d40098a0 tx=0x7f38d402a200 comp rx=0 tx=0).stop 2024-05-08T06:37:10.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.137+0000 7f38e9fb6640 1 -- 172.21.15.163:0/3072653653 shutdown_connections 2024-05-08T06:37:10.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.137+0000 7f38e9fb6640 1 --2- 172.21.15.163:0/3072653653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e40722b0 0x7f38e41427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.137+0000 7f38e9fb6640 1 --2- 172.21.15.163:0/3072653653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e413ff40 0x7f38e4071d70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.139 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.137+0000 7f38e9fb6640 1 -- 172.21.15.163:0/3072653653 >> 172.21.15.163:0/3072653653 conn(0x7f38e40b4da0 msgr2=0x7f38e40b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:10.139 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.137+0000 7f38e9fb6640 1 -- 172.21.15.163:0/3072653653 shutdown_connections 2024-05-08T06:37:10.139 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.137+0000 7f38e9fb6640 1 -- 172.21.15.163:0/3072653653 wait complete. 2024-05-08T06:37:10.139 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.138+0000 7f38e9fb6640 1 Processor -- start 2024-05-08T06:37:10.139 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.138+0000 7f38e9fb6640 1 -- start start 2024-05-08T06:37:10.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e9fb6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e40722b0 0x7f38e40b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e9fb6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38e413ff40 0x7f38e40b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e9fb6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e4142d20 0x7f38e41f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e9fb6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f38e4140320 con 0x7f38e4142d20 2024-05-08T06:37:10.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e9fb6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38e41434c0 con 0x7f38e413ff40 2024-05-08T06:37:10.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e9fb6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f38e41404a0 con 0x7f38e40722b0 2024-05-08T06:37:10.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e3fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e4142d20 0x7f38e41f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e3fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e4142d20 0x7f38e41f6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40458/0 (socket says 172.21.15.163:40458) 2024-05-08T06:37:10.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e3fff640 1 -- 172.21.15.163:0/2436816257 learned_addr learned my addr 172.21.15.163:0/2436816257 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:10.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e37fe640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e40722b0 0x7f38e40b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e2ffd640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38e413ff40 0x7f38e40b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e3fff640 1 -- 172.21.15.163:0/2436816257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e40722b0 msgr2=0x7f38e40b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e3fff640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e40722b0 0x7f38e40b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e3fff640 1 -- 172.21.15.163:0/2436816257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38e413ff40 msgr2=0x7f38e40b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e3fff640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38e413ff40 0x7f38e40b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e3fff640 1 -- 172.21.15.163:0/2436816257 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e4156b40 con 0x7f38e4142d20 2024-05-08T06:37:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e2ffd640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38e413ff40 0x7f38e40b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.139+0000 7f38e3fff640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e4142d20 0x7f38e41f6ce0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00b9a0 tx=0x7f38dc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.140+0000 7f38dbfff640 1 -- 172.21.15.163:0/2436816257 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc019070 con 0x7f38e4142d20 2024-05-08T06:37:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.140+0000 7f38dbfff640 1 -- 172.21.15.163:0/2436816257 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38dc00cba0 con 0x7f38e4142d20 2024-05-08T06:37:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.140+0000 7f38e9fb6640 1 -- 172.21.15.163:0/2436816257 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e415baa0 con 0x7f38e4142d20 2024-05-08T06:37:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.140+0000 7f38dbfff640 1 -- 172.21.15.163:0/2436816257 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc014440 con 0x7f38e4142d20 2024-05-08T06:37:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.140+0000 7f38e9fb6640 1 -- 172.21.15.163:0/2436816257 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e415c060 con 0x7f38e4142d20 2024-05-08T06:37:10.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.141+0000 7f38e9fb6640 1 -- 172.21.15.163:0/2436816257 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e40d3a30 con 0x7f38e4142d20 2024-05-08T06:37:10.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.141+0000 7f38dbfff640 1 -- 172.21.15.163:0/2436816257 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f38dc01c070 con 0x7f38e4142d20 2024-05-08T06:37:10.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.142+0000 7f38dbfff640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f38c40762d0 0x7f38c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.142+0000 7f38dbfff640 1 -- 172.21.15.163:0/2436816257 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f38dc097110 con 0x7f38e4142d20 2024-05-08T06:37:10.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.142+0000 7f38e37fe640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f38c40762d0 0x7f38c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.144 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.142+0000 7f38e37fe640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f38c40762d0 0x7f38c4078790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f38d4034000 tx=0x7f38d402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:10.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.146+0000 7f38dbfff640 1 -- 172.21.15.163:0/2436816257 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f38e40d3a30 con 0x7f38e4142d20 2024-05-08T06:37:10.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.341+0000 7f38e9fb6640 1 -- 172.21.15.163:0/2436816257 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f38e40b2aa0 con 0x7f38e4142d20 2024-05-08T06:37:10.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.341+0000 7f38dbfff640 1 -- 172.21.15.163:0/2436816257 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f38e40b2aa0 con 0x7f38e4142d20 2024-05-08T06:37:10.341 INFO:teuthology.orchestra.run.smithi163.stderr:listed 0 entries 2024-05-08T06:37:10.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.343+0000 7f38e9fb6640 1 -- 172.21.15.163:0/2436816257 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f38c40762d0 msgr2=0x7f38c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.343+0000 7f38e9fb6640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f38c40762d0 0x7f38c4078790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f38d4034000 tx=0x7f38d402e000 comp rx=0 tx=0).stop 2024-05-08T06:37:10.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.343+0000 7f38e9fb6640 1 -- 172.21.15.163:0/2436816257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e4142d20 msgr2=0x7f38e41f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.343+0000 7f38e9fb6640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e4142d20 0x7f38e41f6ce0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00b9a0 tx=0x7f38dc00be60 comp rx=0 tx=0).stop 2024-05-08T06:37:10.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.343+0000 7f38e9fb6640 1 -- 172.21.15.163:0/2436816257 shutdown_connections 2024-05-08T06:37:10.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.343+0000 7f38e9fb6640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f38c40762d0 0x7f38c4078790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.343+0000 7f38e9fb6640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e4142d20 0x7f38e41f6ce0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.343+0000 7f38e9fb6640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38e413ff40 0x7f38e40b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.343+0000 7f38e9fb6640 1 --2- 172.21.15.163:0/2436816257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e40722b0 0x7f38e40b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.343+0000 7f38e9fb6640 1 -- 172.21.15.163:0/2436816257 >> 172.21.15.163:0/2436816257 conn(0x7f38e40b4da0 msgr2=0x7f38e40d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:10.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.343+0000 7f38e9fb6640 1 -- 172.21.15.163:0/2436816257 shutdown_connections 2024-05-08T06:37:10.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.343+0000 7f38e9fb6640 1 -- 172.21.15.163:0/2436816257 wait complete. 2024-05-08T06:37:10.355 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-08T06:37:10.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:10.355 DEBUG:teuthology.orchestra.run.smithi192:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-08T06:37:10.386 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T06:37:10.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:10.387 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T06:37:10.445 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T06:37:10.446 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:10.547 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.547+0000 7f6f50f9e640 1 Processor -- start 2024-05-08T06:37:10.547 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.547+0000 7f6f50f9e640 1 -- start start 2024-05-08T06:37:10.548 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f50f9e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f4c0b8840 0x7f6f4c06de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f50f9e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6f4c0d3940 0x7f6f4c09ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f50f9e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f4c0cf240 con 0x7f6f4c09bf50 2024-05-08T06:37:10.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f50f9e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6f4c06e350 con 0x7f6f4c0d3940 2024-05-08T06:37:10.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f50f9e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4c06e5b0 con 0x7f6f4c0b8840 2024-05-08T06:37:10.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f4b7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f4c0b8840 0x7f6f4c06de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f4affd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6f4c0d3940 0x7f6f4c09ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f4b7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f4c0b8840 0x7f6f4c06de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:58456/0 (socket says 172.21.15.163:58456) 2024-05-08T06:37:10.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f4bfff640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f6f4c09bf50 0x7f6f4c0d0fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:44204/0 (socket says 172.21.15.163:44204) 2024-05-08T06:37:10.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f4b7fe640 1 -- 172.21.15.163:0/415694412 learned_addr learned my addr 172.21.15.163:0/415694412 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:10.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f4affd640 1 -- 172.21.15.163:0/415694412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f4c0b8840 msgr2=0x7f6f4c06de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:10.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f4affd640 1 --2- 172.21.15.163:0/415694412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f4c0b8840 0x7f6f4c06de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f4affd640 1 -- 172.21.15.163:0/415694412 >> v1:172.21.15.192:6789/0 conn(0x7f6f4c09bf50 legacy=0x7f6f4c0d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f4affd640 1 -- 172.21.15.163:0/415694412 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c0cd470 con 0x7f6f4c0d3940 2024-05-08T06:37:10.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f4a7fc640 1 -- 172.21.15.163:0/415694412 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2633198214 0 0) 0x7f6f4c0cf240 con 0x7f6f4c09bf50 2024-05-08T06:37:10.550 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.548+0000 7f6f4b7fe640 1 --2- 172.21.15.163:0/415694412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f4c0b8840 0x7f6f4c06de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:10.551 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.549+0000 7f6f4affd640 1 --2- 172.21.15.163:0/415694412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f4c0d3940 0x7f6f4c09ba10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6f340097a0 tx=0x7f6f3402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a5035c846b6991b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:10.551 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.549+0000 7f6f4a7fc640 1 -- 172.21.15.163:0/415694412 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34037070 con 0x7f6f4c0d3940 2024-05-08T06:37:10.551 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.549+0000 7f6f4a7fc640 1 -- 172.21.15.163:0/415694412 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f34032440 con 0x7f6f4c0d3940 2024-05-08T06:37:10.551 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.549+0000 7f6f4a7fc640 1 -- 172.21.15.163:0/415694412 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34032810 con 0x7f6f4c0d3940 2024-05-08T06:37:10.551 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.550+0000 7f6f50f9e640 1 -- 172.21.15.163:0/415694412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f4c0d3940 msgr2=0x7f6f4c09ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.551 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.550+0000 7f6f50f9e640 1 --2- 172.21.15.163:0/415694412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f4c0d3940 0x7f6f4c09ba10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6f340097a0 tx=0x7f6f3402a180 comp rx=0 tx=0).stop 2024-05-08T06:37:10.551 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.550+0000 7f6f50f9e640 1 -- 172.21.15.163:0/415694412 shutdown_connections 2024-05-08T06:37:10.551 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.550+0000 7f6f50f9e640 1 --2- 172.21.15.163:0/415694412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f4c0d3940 0x7f6f4c09ba10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.551 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.550+0000 7f6f50f9e640 1 --2- 172.21.15.163:0/415694412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f4c0b8840 0x7f6f4c06de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.551 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.550+0000 7f6f50f9e640 1 -- 172.21.15.163:0/415694412 >> 172.21.15.163:0/415694412 conn(0x7f6f4c0d2e20 msgr2=0x7f6f4c0d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:10.551 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.550+0000 7f6f50f9e640 1 -- 172.21.15.163:0/415694412 shutdown_connections 2024-05-08T06:37:10.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.550+0000 7f6f50f9e640 1 -- 172.21.15.163:0/415694412 wait complete. 2024-05-08T06:37:10.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.551+0000 7f6f50f9e640 1 Processor -- start 2024-05-08T06:37:10.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.551+0000 7f6f50f9e640 1 -- start start 2024-05-08T06:37:10.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.552+0000 7f6f50f9e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f4c09bf50 0x7f6f4c157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.552+0000 7f6f50f9e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f4c0b8840 0x7f6f4c0ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.552+0000 7f6f50f9e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f4c0d3940 0x7f6f4c1f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.552+0000 7f6f50f9e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c0cdd10 con 0x7f6f4c09bf50 2024-05-08T06:37:10.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.552+0000 7f6f50f9e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c0cda30 con 0x7f6f4c0b8840 2024-05-08T06:37:10.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.552+0000 7f6f50f9e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4c0b5d10 con 0x7f6f4c0d3940 2024-05-08T06:37:10.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.552+0000 7f6f4b7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f4c09bf50 0x7f6f4c157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.552+0000 7f6f4b7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f4c09bf50 0x7f6f4c157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40474/0 (socket says 172.21.15.163:40474) 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.552+0000 7f6f4b7fe640 1 -- 172.21.15.163:0/769506882 learned_addr learned my addr 172.21.15.163:0/769506882 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.552+0000 7f6f4affd640 1 --2- 172.21.15.163:0/769506882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f4c0b8840 0x7f6f4c0ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.552+0000 7f6f4bfff640 1 --2- 172.21.15.163:0/769506882 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f4c0d3940 0x7f6f4c1f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.553+0000 7f6f4b7fe640 1 -- 172.21.15.163:0/769506882 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f4c0d3940 msgr2=0x7f6f4c1f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.553+0000 7f6f4b7fe640 1 --2- 172.21.15.163:0/769506882 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f4c0d3940 0x7f6f4c1f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.553+0000 7f6f4b7fe640 1 -- 172.21.15.163:0/769506882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f4c0b8840 msgr2=0x7f6f4c0ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.553+0000 7f6f4b7fe640 1 --2- 172.21.15.163:0/769506882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f4c0b8840 0x7f6f4c0ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.553+0000 7f6f4b7fe640 1 -- 172.21.15.163:0/769506882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c157940 con 0x7f6f4c09bf50 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.553+0000 7f6f4b7fe640 1 --2- 172.21.15.163:0/769506882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f4c09bf50 0x7f6f4c157400 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00ea00 tx=0x7f6f3c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.553+0000 7f6f43fff640 1 -- 172.21.15.163:0/769506882 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c017070 con 0x7f6f4c09bf50 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.553+0000 7f6f43fff640 1 -- 172.21.15.163:0/769506882 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f3c00ca60 con 0x7f6f4c09bf50 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.553+0000 7f6f50f9e640 1 -- 172.21.15.163:0/769506882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c1f5f70 con 0x7f6f4c09bf50 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.553+0000 7f6f43fff640 1 -- 172.21.15.163:0/769506882 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00ce10 con 0x7f6f4c09bf50 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.553+0000 7f6f50f9e640 1 -- 172.21.15.163:0/769506882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c0ca100 con 0x7f6f4c09bf50 2024-05-08T06:37:10.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.554+0000 7f6f50f9e640 1 -- 172.21.15.163:0/769506882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4c06a3d0 con 0x7f6f4c09bf50 2024-05-08T06:37:10.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.555+0000 7f6f43fff640 1 -- 172.21.15.163:0/769506882 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6f3c01a070 con 0x7f6f4c09bf50 2024-05-08T06:37:10.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.555+0000 7f6f43fff640 1 --2- 172.21.15.163:0/769506882 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6f2c076300 0x7f6f2c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.555+0000 7f6f43fff640 1 -- 172.21.15.163:0/769506882 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6f3c0975a0 con 0x7f6f4c09bf50 2024-05-08T06:37:10.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.555+0000 7f6f4affd640 1 --2- 172.21.15.163:0/769506882 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6f2c076300 0x7f6f2c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.555+0000 7f6f4affd640 1 --2- 172.21.15.163:0/769506882 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6f2c076300 0x7f6f2c0787c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6f34034000 tx=0x7f6f3402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:10.559 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.560+0000 7f6f43fff640 1 -- 172.21.15.163:0/769506882 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6f4c157940 con 0x7f6f4c09bf50 2024-05-08T06:37:10.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.779+0000 7f6f50f9e640 1 -- 172.21.15.163:0/769506882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f4c069770 con 0x7f6f4c09bf50 2024-05-08T06:37:10.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.779+0000 7f6f43fff640 1 -- 172.21.15.163:0/769506882 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f6f3c067e20 con 0x7f6f4c09bf50 2024-05-08T06:37:10.779 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:10.780 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2413407842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2413407842},{"type":"v1","addr":"172.21.15.163:6835","nonce":2413407842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2405203133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2405203133},{"type":"v1","addr":"172.21.15.163:6837","nonce":2405203133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3301087897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3301087897},{"type":"v1","addr":"172.21.15.192:6833","nonce":3301087897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:36:40.985978+0000","modified":"2024-05-08T06:36:43.900778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/354782184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":354782184},{"type":"v1","addr":"172.21.15.192:6835","nonce":354782184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-05-08T06:37:10.780 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 7 2024-05-08T06:37:10.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.781+0000 7f6f50f9e640 1 -- 172.21.15.163:0/769506882 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6f2c076300 msgr2=0x7f6f2c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.781+0000 7f6f50f9e640 1 --2- 172.21.15.163:0/769506882 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6f2c076300 0x7f6f2c0787c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6f34034000 tx=0x7f6f3402e000 comp rx=0 tx=0).stop 2024-05-08T06:37:10.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.781+0000 7f6f50f9e640 1 -- 172.21.15.163:0/769506882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f4c09bf50 msgr2=0x7f6f4c157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.781+0000 7f6f50f9e640 1 --2- 172.21.15.163:0/769506882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f4c09bf50 0x7f6f4c157400 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00ea00 tx=0x7f6f3c00eec0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.781+0000 7f6f50f9e640 1 -- 172.21.15.163:0/769506882 shutdown_connections 2024-05-08T06:37:10.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.781+0000 7f6f50f9e640 1 --2- 172.21.15.163:0/769506882 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6f2c076300 0x7f6f2c0787c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.781+0000 7f6f50f9e640 1 --2- 172.21.15.163:0/769506882 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f4c0d3940 0x7f6f4c1f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.781+0000 7f6f50f9e640 1 --2- 172.21.15.163:0/769506882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f4c0b8840 0x7f6f4c0ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.781+0000 7f6f50f9e640 1 --2- 172.21.15.163:0/769506882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f4c09bf50 0x7f6f4c157400 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.781+0000 7f6f50f9e640 1 -- 172.21.15.163:0/769506882 >> 172.21.15.163:0/769506882 conn(0x7f6f4c0d2e20 msgr2=0x7f6f4c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:10.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.781+0000 7f6f50f9e640 1 -- 172.21.15.163:0/769506882 shutdown_connections 2024-05-08T06:37:10.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.781+0000 7f6f50f9e640 1 -- 172.21.15.163:0/769506882 wait complete. 2024-05-08T06:37:10.793 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:10.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.893+0000 7fb54bbc1640 1 Processor -- start 2024-05-08T06:37:10.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.894+0000 7fb54bbc1640 1 -- start start 2024-05-08T06:37:10.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.894+0000 7fb54bbc1640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb5440b4340 0x7fb5440b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.894+0000 7fb54bbc1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5440741d0 con 0x7fb5440b0430 2024-05-08T06:37:10.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.894+0000 7fb54bbc1640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5440b5610 con 0x7fb5440b0810 2024-05-08T06:37:10.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.894+0000 7fb54bbc1640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb5440b1900 con 0x7fb5440b4340 2024-05-08T06:37:10.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54995d640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb5440b0430 0x7fb54415f9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60554/0 (socket says 172.21.15.163:60554) 2024-05-08T06:37:10.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54995d640 1 -- 172.21.15.163:0/916875574 learned_addr learned my addr 172.21.15.163:0/916875574 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:10.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54a15e640 1 --2- 172.21.15.163:0/916875574 >> v2:172.21.15.192:3300/0 conn(0x7fb5440b4340 0x7fb5440b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 -- 172.21.15.163:0/916875574 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3966101792 0 0) 0x7fb5440741d0 con 0x7fb5440b0430 2024-05-08T06:37:10.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 -- 172.21.15.163:0/916875574 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb530003520 con 0x7fb5440b0430 2024-05-08T06:37:10.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 -- 172.21.15.163:0/916875574 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2877048218 0 0) 0x7fb530003520 con 0x7fb5440b0430 2024-05-08T06:37:10.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 -- 172.21.15.163:0/916875574 >> v2:172.21.15.192:3300/0 conn(0x7fb5440b4340 msgr2=0x7fb5440b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:10.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 --2- 172.21.15.163:0/916875574 >> v2:172.21.15.192:3300/0 conn(0x7fb5440b4340 0x7fb5440b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 -- 172.21.15.163:0/916875574 >> v1:172.21.15.192:6789/0 conn(0x7fb5440b0810 legacy=0x7fb5440b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 -- 172.21.15.163:0/916875574 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5440b6fd0 con 0x7fb5440b0430 2024-05-08T06:37:10.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 -- 172.21.15.163:0/916875574 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7fb540001a70 con 0x7fb5440b0430 2024-05-08T06:37:10.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 -- 172.21.15.163:0/916875574 >> v1:172.21.15.163:6789/0 conn(0x7fb5440b0430 legacy=0x7fb54415f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 --2- 172.21.15.163:0/916875574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb530003a00 0x7fb530003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 --2- 172.21.15.163:0/916875574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb530004800 0x7fb530004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 --2- 172.21.15.163:0/916875574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5300051c0 0x7fb53000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 -- 172.21.15.163:0/916875574 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb53000c0e0 con 0x7fb5300051c0 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 -- 172.21.15.163:0/916875574 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb53000c290 con 0x7fb530004800 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54895b640 1 -- 172.21.15.163:0/916875574 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb53000c490 con 0x7fb530003a00 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54a15e640 1 --2- 172.21.15.163:0/916875574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5300051c0 0x7fb53000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54995d640 1 --2- 172.21.15.163:0/916875574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb530003a00 0x7fb530003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.895+0000 7fb54915c640 1 --2- 172.21.15.163:0/916875574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb530004800 0x7fb530004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.896+0000 7fb54a15e640 1 -- 172.21.15.163:0/916875574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb530003a00 msgr2=0x7fb530003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.896+0000 7fb54a15e640 1 --2- 172.21.15.163:0/916875574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb530003a00 0x7fb530003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.896+0000 7fb54a15e640 1 -- 172.21.15.163:0/916875574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb530004800 msgr2=0x7fb530004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.896+0000 7fb54a15e640 1 --2- 172.21.15.163:0/916875574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb530004800 0x7fb530004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.896+0000 7fb54a15e640 1 -- 172.21.15.163:0/916875574 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb530003520 con 0x7fb5300051c0 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.896+0000 7fb54a15e640 1 -- 172.21.15.163:0/916875574 reap_dead start 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.896+0000 7fb54a15e640 1 --2- 172.21.15.163:0/916875574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5300051c0 0x7fb53000b9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009ac0 tx=0x7fb530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee582c9c0484f75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.896+0000 7fb54895b640 1 -- 172.21.15.163:0/916875574 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c009690 con 0x7fb5300051c0 2024-05-08T06:37:10.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.896+0000 7fb54895b640 1 -- 172.21.15.163:0/916875574 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb53c004020 con 0x7fb5300051c0 2024-05-08T06:37:10.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.896+0000 7fb54895b640 1 -- 172.21.15.163:0/916875574 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0043d0 con 0x7fb5300051c0 2024-05-08T06:37:10.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.897+0000 7fb54bbc1640 1 -- 172.21.15.163:0/916875574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5300051c0 msgr2=0x7fb53000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.897+0000 7fb54bbc1640 1 --2- 172.21.15.163:0/916875574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5300051c0 0x7fb53000b9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009ac0 tx=0x7fb530005830 comp rx=0 tx=0).stop 2024-05-08T06:37:10.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.897+0000 7fb54bbc1640 1 -- 172.21.15.163:0/916875574 shutdown_connections 2024-05-08T06:37:10.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.897+0000 7fb54bbc1640 1 --2- 172.21.15.163:0/916875574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5300051c0 0x7fb53000b9c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.897+0000 7fb54bbc1640 1 -- 172.21.15.163:0/916875574 >> 172.21.15.163:0/916875574 conn(0x7fb54406d430 msgr2=0x7fb54406d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:10.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.897+0000 7fb54bbc1640 1 -- 172.21.15.163:0/916875574 shutdown_connections 2024-05-08T06:37:10.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.897+0000 7fb54bbc1640 1 -- 172.21.15.163:0/916875574 wait complete. 2024-05-08T06:37:10.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.898+0000 7fb54bbc1640 1 Processor -- start 2024-05-08T06:37:10.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.898+0000 7fb54bbc1640 1 -- start start 2024-05-08T06:37:10.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54bbc1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5300051c0 0x7fb54415fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54bbc1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb544157480 0x7fb544153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54bbc1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb544161270 0x7fb54415d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54bbc1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb5440b6db0 con 0x7fb544157480 2024-05-08T06:37:10.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54bbc1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5440b6c30 con 0x7fb544161270 2024-05-08T06:37:10.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54bbc1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb544070ea0 con 0x7fb5300051c0 2024-05-08T06:37:10.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54915c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb544157480 0x7fb544153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54915c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb544157480 0x7fb544153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40498/0 (socket says 172.21.15.163:40498) 2024-05-08T06:37:10.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54915c640 1 -- 172.21.15.163:0/1464920664 learned_addr learned my addr 172.21.15.163:0/1464920664 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:10.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54a15e640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb544161270 0x7fb54415d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54995d640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5300051c0 0x7fb54415fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54915c640 1 -- 172.21.15.163:0/1464920664 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5300051c0 msgr2=0x7fb54415fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54915c640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5300051c0 0x7fb54415fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54915c640 1 -- 172.21.15.163:0/1464920664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb544161270 msgr2=0x7fb54415d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54915c640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb544161270 0x7fb54415d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54915c640 1 -- 172.21.15.163:0/1464920664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54415c930 con 0x7fb544157480 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54995d640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5300051c0 0x7fb54415fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.899+0000 7fb54a15e640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb544161270 0x7fb54415d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.900+0000 7fb54915c640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb544157480 0x7fb544153a80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb53400f800 tx=0x7fb53400fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.900+0000 7fb53a7fc640 1 -- 172.21.15.163:0/1464920664 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53401a070 con 0x7fb544157480 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.900+0000 7fb53a7fc640 1 -- 172.21.15.163:0/1464920664 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb534005320 con 0x7fb544157480 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.900+0000 7fb53a7fc640 1 -- 172.21.15.163:0/1464920664 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb534007270 con 0x7fb544157480 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.900+0000 7fb54bbc1640 1 -- 172.21.15.163:0/1464920664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5441601d0 con 0x7fb544157480 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.900+0000 7fb54bbc1640 1 -- 172.21.15.163:0/1464920664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb544160790 con 0x7fb544157480 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.901+0000 7fb54bbc1640 1 -- 172.21.15.163:0/1464920664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb54406a550 con 0x7fb544157480 2024-05-08T06:37:10.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.901+0000 7fb53a7fc640 1 -- 172.21.15.163:0/1464920664 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb53401d070 con 0x7fb544157480 2024-05-08T06:37:10.905 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.902+0000 7fb53a7fc640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb5280762d0 0x7fb528078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:10.905 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.902+0000 7fb53a7fc640 1 -- 172.21.15.163:0/1464920664 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb534096d60 con 0x7fb544157480 2024-05-08T06:37:10.905 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.902+0000 7fb54995d640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb5280762d0 0x7fb528078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:10.905 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.902+0000 7fb54995d640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb5280762d0 0x7fb528078790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb540001bd0 tx=0x7fb540006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:10.906 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:10.907+0000 7fb53a7fc640 1 -- 172.21.15.163:0/1464920664 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb5340606c0 con 0x7fb544157480 2024-05-08T06:37:11.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.122+0000 7fb54bbc1640 1 -- 172.21.15.163:0/1464920664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb544073d40 con 0x7fb544157480 2024-05-08T06:37:11.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.122+0000 7fb53a7fc640 1 -- 172.21.15.163:0/1464920664 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7fb53405ffc0 con 0x7fb544157480 2024-05-08T06:37:11.123 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:11.123 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2413407842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2413407842},{"type":"v1","addr":"172.21.15.163:6835","nonce":2413407842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2405203133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2405203133},{"type":"v1","addr":"172.21.15.163:6837","nonce":2405203133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3301087897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3301087897},{"type":"v1","addr":"172.21.15.192:6833","nonce":3301087897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:36:40.985978+0000","modified":"2024-05-08T06:36:43.900778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/354782184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":354782184},{"type":"v1","addr":"172.21.15.192:6835","nonce":354782184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-05-08T06:37:11.123 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 7 2024-05-08T06:37:11.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.124+0000 7fb54bbc1640 1 -- 172.21.15.163:0/1464920664 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb5280762d0 msgr2=0x7fb528078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.124+0000 7fb54bbc1640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb5280762d0 0x7fb528078790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb540001bd0 tx=0x7fb540006040 comp rx=0 tx=0).stop 2024-05-08T06:37:11.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.124+0000 7fb54bbc1640 1 -- 172.21.15.163:0/1464920664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb544157480 msgr2=0x7fb544153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.124+0000 7fb54bbc1640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb544157480 0x7fb544153a80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb53400f800 tx=0x7fb53400fcc0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.124+0000 7fb54bbc1640 1 -- 172.21.15.163:0/1464920664 shutdown_connections 2024-05-08T06:37:11.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.124+0000 7fb54bbc1640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb5280762d0 0x7fb528078790 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.124+0000 7fb54bbc1640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb544161270 0x7fb54415d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.124+0000 7fb54bbc1640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb544157480 0x7fb544153a80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.124+0000 7fb54bbc1640 1 --2- 172.21.15.163:0/1464920664 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5300051c0 0x7fb54415fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.124+0000 7fb54bbc1640 1 -- 172.21.15.163:0/1464920664 >> 172.21.15.163:0/1464920664 conn(0x7fb54406d430 msgr2=0x7fb54406fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:11.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.124+0000 7fb54bbc1640 1 -- 172.21.15.163:0/1464920664 shutdown_connections 2024-05-08T06:37:11.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.124+0000 7fb54bbc1640 1 -- 172.21.15.163:0/1464920664 wait complete. 2024-05-08T06:37:11.136 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:37:11.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.235+0000 7f36e2fda640 1 Processor -- start 2024-05-08T06:37:11.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.235+0000 7f36e2fda640 1 -- start start 2024-05-08T06:37:11.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e2fda640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f36dc142390 0x7f36dc0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e2fda640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36dc1636d0 0x7f36dc142a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e2fda640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36dc142770 con 0x7f36dc0b65e0 2024-05-08T06:37:11.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e2fda640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f36dc163d30 con 0x7f36dc142390 2024-05-08T06:37:11.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e2fda640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36dc15f990 con 0x7f36dc1636d0 2024-05-08T06:37:11.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d3fff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f36dc0b65e0 0x7f36dc162fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60558/0 (socket says 172.21.15.163:60558) 2024-05-08T06:37:11.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d3fff640 1 -- 172.21.15.163:0/1860304907 learned_addr learned my addr 172.21.15.163:0/1860304907 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:11.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e0d76640 1 --2- 172.21.15.163:0/1860304907 >> v2:172.21.15.192:3300/0 conn(0x7f36dc142390 0x7f36dc0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e1577640 1 --2- 172.21.15.163:0/1860304907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36dc1636d0 0x7f36dc142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 -- 172.21.15.163:0/1860304907 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3981337327 0 0) 0x7f36dc142770 con 0x7f36dc0b65e0 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 -- 172.21.15.163:0/1860304907 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36c4003520 con 0x7f36dc0b65e0 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 -- 172.21.15.163:0/1860304907 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3388041541 0 0) 0x7f36c4003520 con 0x7f36dc0b65e0 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 -- 172.21.15.163:0/1860304907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36dc1636d0 msgr2=0x7f36dc142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 --2- 172.21.15.163:0/1860304907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36dc1636d0 0x7f36dc142a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 -- 172.21.15.163:0/1860304907 >> v2:172.21.15.192:3300/0 conn(0x7f36dc142390 msgr2=0x7f36dc0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 --2- 172.21.15.163:0/1860304907 >> v2:172.21.15.192:3300/0 conn(0x7f36dc142390 0x7f36dc0b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 -- 172.21.15.163:0/1860304907 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36dc15fc70 con 0x7f36dc0b65e0 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 -- 172.21.15.163:0/1860304907 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f36cc0016f0 con 0x7f36dc0b65e0 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 -- 172.21.15.163:0/1860304907 >> v1:172.21.15.163:6789/0 conn(0x7f36dc0b65e0 legacy=0x7f36dc162fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 --2- 172.21.15.163:0/1860304907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c4003a00 0x7f36c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 --2- 172.21.15.163:0/1860304907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36dc1636d0 0x7f36c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 --2- 172.21.15.163:0/1860304907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36c4004d40 0x7f36c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 -- 172.21.15.163:0/1860304907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc163d30 con 0x7f36c4003a00 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 -- 172.21.15.163:0/1860304907 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc15f990 con 0x7f36c4004d40 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d37fe640 1 -- 172.21.15.163:0/1860304907 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36c400b9c0 con 0x7f36dc1636d0 2024-05-08T06:37:11.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e0d76640 1 --2- 172.21.15.163:0/1860304907 >> v2:172.21.15.192:3300/0 conn(0x7f36dc142390 0x7f36dc0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e0d76640 1 --2- 172.21.15.163:0/1860304907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c4003a00 0x7f36c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36d3fff640 1 --2- 172.21.15.163:0/1860304907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36dc1636d0 0x7f36c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e1577640 1 --2- 172.21.15.163:0/1860304907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36c4004d40 0x7f36c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e0d76640 1 -- 172.21.15.163:0/1860304907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36dc1636d0 msgr2=0x7f36c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e0d76640 1 --2- 172.21.15.163:0/1860304907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36dc1636d0 0x7f36c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e0d76640 1 -- 172.21.15.163:0/1860304907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36c4004d40 msgr2=0x7f36c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e0d76640 1 --2- 172.21.15.163:0/1860304907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36c4004d40 0x7f36c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.237+0000 7f36e0d76640 1 -- 172.21.15.163:0/1860304907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c4003520 con 0x7f36c4003a00 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.238+0000 7f36e0d76640 1 --2- 172.21.15.163:0/1860304907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c4003a00 0x7f36c4003e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f36d800e350 tx=0x7f36c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a338da6f469a9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.238+0000 7f36d37fe640 1 -- 172.21.15.163:0/1860304907 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8009dc0 con 0x7f36c4003a00 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.238+0000 7f36d37fe640 1 -- 172.21.15.163:0/1860304907 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36d8003b90 con 0x7f36c4003a00 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.238+0000 7f36d37fe640 1 -- 172.21.15.163:0/1860304907 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8004020 con 0x7f36c4003a00 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.238+0000 7f36e2fda640 1 -- 172.21.15.163:0/1860304907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c4003a00 msgr2=0x7f36c4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.238+0000 7f36e2fda640 1 --2- 172.21.15.163:0/1860304907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c4003a00 0x7f36c4003e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f36d800e350 tx=0x7f36c4005830 comp rx=0 tx=0).stop 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.238+0000 7f36e1577640 1 -- 172.21.15.163:0/1860304907 reap_dead start 2024-05-08T06:37:11.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.238+0000 7f36e2fda640 1 -- 172.21.15.163:0/1860304907 shutdown_connections 2024-05-08T06:37:11.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.238+0000 7f36e2fda640 1 -- 172.21.15.163:0/1860304907 >> 172.21.15.163:0/1860304907 conn(0x7f36dc0b4da0 msgr2=0x7f36dc0b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:11.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.239+0000 7f36e2fda640 1 -- 172.21.15.163:0/1860304907 shutdown_connections 2024-05-08T06:37:11.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.239+0000 7f36e2fda640 1 -- 172.21.15.163:0/1860304907 wait complete. 2024-05-08T06:37:11.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.240+0000 7f36e2fda640 1 Processor -- start 2024-05-08T06:37:11.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.240+0000 7f36e2fda640 1 -- start start 2024-05-08T06:37:11.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e2fda640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c4003a00 0x7f36dc0ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e2fda640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36dc161cc0 0x7f36dc1e9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e2fda640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36dc162140 0x7f36dc140670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e2fda640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc160750 con 0x7f36c4003a00 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e2fda640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc160230 con 0x7f36dc162140 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e2fda640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36dc142a70 con 0x7f36dc161cc0 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e0d76640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c4003a00 0x7f36dc0ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e0d76640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c4003a00 0x7f36dc0ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40516/0 (socket says 172.21.15.163:40516) 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e0d76640 1 -- 172.21.15.163:0/2412753030 learned_addr learned my addr 172.21.15.163:0/2412753030 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e1577640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36dc162140 0x7f36dc140670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36d3fff640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36dc161cc0 0x7f36dc1e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e0d76640 1 -- 172.21.15.163:0/2412753030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36dc161cc0 msgr2=0x7f36dc1e9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e0d76640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36dc161cc0 0x7f36dc1e9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e0d76640 1 -- 172.21.15.163:0/2412753030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36dc162140 msgr2=0x7f36dc140670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e0d76640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36dc162140 0x7f36dc140670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e0d76640 1 -- 172.21.15.163:0/2412753030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36dc1ea0b0 con 0x7f36c4003a00 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36d3fff640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36dc161cc0 0x7f36dc1e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e0d76640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c4003a00 0x7f36dc0ceca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f36d800e470 tx=0x7f36d8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36d17fa640 1 -- 172.21.15.163:0/2412753030 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8046070 con 0x7f36c4003a00 2024-05-08T06:37:11.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36d17fa640 1 -- 172.21.15.163:0/2412753030 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36d8036610 con 0x7f36c4003a00 2024-05-08T06:37:11.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e2fda640 1 -- 172.21.15.163:0/2412753030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36dc0ce1c0 con 0x7f36c4003a00 2024-05-08T06:37:11.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36d17fa640 1 -- 172.21.15.163:0/2412753030 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8009dc0 con 0x7f36c4003a00 2024-05-08T06:37:11.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.241+0000 7f36e2fda640 1 -- 172.21.15.163:0/2412753030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36dc0cf1e0 con 0x7f36c4003a00 2024-05-08T06:37:11.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.243+0000 7f36e2fda640 1 -- 172.21.15.163:0/2412753030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36dc0ce390 con 0x7f36c4003a00 2024-05-08T06:37:11.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.243+0000 7f36d17fa640 1 -- 172.21.15.163:0/2412753030 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f36d8049070 con 0x7f36c4003a00 2024-05-08T06:37:11.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.243+0000 7f36d17fa640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f36c8076300 0x7f36c80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.244+0000 7f36d17fa640 1 -- 172.21.15.163:0/2412753030 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f36d80b8130 con 0x7f36c4003a00 2024-05-08T06:37:11.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.244+0000 7f36d3fff640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f36c8076300 0x7f36c80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.244+0000 7f36d3fff640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f36c8076300 0x7f36c80787c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f36cc001010 tx=0x7f36cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:11.248 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.249+0000 7f36d17fa640 1 -- 172.21.15.163:0/2412753030 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f36d8032030 con 0x7f36c4003a00 2024-05-08T06:37:11.432 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.432+0000 7f36e2fda640 1 -- 172.21.15.163:0/2412753030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36dc075dc0 con 0x7f36c4003a00 2024-05-08T06:37:11.433 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.433+0000 7f36d17fa640 1 -- 172.21.15.163:0/2412753030 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f36d8081a90 con 0x7f36c4003a00 2024-05-08T06:37:11.434 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:11.434 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:36:42.641260+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:36:38.853461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:36:39.952926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:37:11.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.435+0000 7f36e2fda640 1 -- 172.21.15.163:0/2412753030 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f36c8076300 msgr2=0x7f36c80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.435+0000 7f36e2fda640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f36c8076300 0x7f36c80787c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f36cc001010 tx=0x7f36cc009040 comp rx=0 tx=0).stop 2024-05-08T06:37:11.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.435+0000 7f36e2fda640 1 -- 172.21.15.163:0/2412753030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c4003a00 msgr2=0x7f36dc0ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.435+0000 7f36e2fda640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c4003a00 0x7f36dc0ceca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f36d800e470 tx=0x7f36d8009750 comp rx=0 tx=0).stop 2024-05-08T06:37:11.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.435+0000 7f36e2fda640 1 -- 172.21.15.163:0/2412753030 shutdown_connections 2024-05-08T06:37:11.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.435+0000 7f36e2fda640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f36c8076300 0x7f36c80787c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.435+0000 7f36e2fda640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36dc162140 0x7f36dc140670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.435+0000 7f36e2fda640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36dc161cc0 0x7f36dc1e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.435+0000 7f36e2fda640 1 --2- 172.21.15.163:0/2412753030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c4003a00 0x7f36dc0ceca0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.436+0000 7f36e2fda640 1 -- 172.21.15.163:0/2412753030 >> 172.21.15.163:0/2412753030 conn(0x7f36dc0b4da0 msgr2=0x7f36dc0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:11.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.436+0000 7f36e2fda640 1 -- 172.21.15.163:0/2412753030 shutdown_connections 2024-05-08T06:37:11.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.436+0000 7f36e2fda640 1 -- 172.21.15.163:0/2412753030 wait complete. 2024-05-08T06:37:11.447 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T06:37:11.447 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:11.548 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.548+0000 7f8cd62cf640 1 Processor -- start 2024-05-08T06:37:11.548 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.548+0000 7f8cd62cf640 1 -- start start 2024-05-08T06:37:11.551 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.549+0000 7f8cd62cf640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8cd00d3a00 0x7f8cd0144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.551 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.549+0000 7f8cd62cf640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cd015fca0 con 0x7f8cd0163800 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.549+0000 7f8cd62cf640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cd015ff80 con 0x7f8cd01423c0 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.549+0000 7f8cd62cf640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8cd01608b0 con 0x7f8cd00d3a00 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.549+0000 7f8cc7fff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8cd0163800 0x7f8cd00d32f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60572/0 (socket says 172.21.15.163:60572) 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.549+0000 7f8cc7fff640 1 -- 172.21.15.163:0/772555495 learned_addr learned my addr 172.21.15.163:0/772555495 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 -- 172.21.15.163:0/772555495 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2062196723 0 0) 0x7f8cd015fca0 con 0x7f8cd0163800 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 -- 172.21.15.163:0/772555495 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cbc003520 con 0x7f8cd0163800 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8cd486c640 1 --2- 172.21.15.163:0/772555495 >> v2:172.21.15.192:3300/0 conn(0x7f8cd00d3a00 0x7f8cd0144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 -- 172.21.15.163:0/772555495 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 141511265 0 0) 0x7f8cbc003520 con 0x7f8cd0163800 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 -- 172.21.15.163:0/772555495 >> v2:172.21.15.192:3300/0 conn(0x7f8cd00d3a00 msgr2=0x7f8cd0144dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 --2- 172.21.15.163:0/772555495 >> v2:172.21.15.192:3300/0 conn(0x7f8cd00d3a00 0x7f8cd0144dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 -- 172.21.15.163:0/772555495 >> v1:172.21.15.192:6789/0 conn(0x7f8cd01423c0 legacy=0x7f8cd0142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 -- 172.21.15.163:0/772555495 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd0160260 con 0x7f8cd0163800 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 -- 172.21.15.163:0/772555495 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f8cb80016f0 con 0x7f8cd0163800 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 -- 172.21.15.163:0/772555495 >> v1:172.21.15.163:6789/0 conn(0x7f8cd0163800 legacy=0x7f8cd00d32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 --2- 172.21.15.163:0/772555495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc003a50 0x7f8cbc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 --2- 172.21.15.163:0/772555495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cbc004800 0x7f8cbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 --2- 172.21.15.163:0/772555495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cbc0051c0 0x7f8cbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 -- 172.21.15.163:0/772555495 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd01608b0 con 0x7f8cbc004800 2024-05-08T06:37:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 -- 172.21.15.163:0/772555495 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc00c160 con 0x7f8cbc0051c0 2024-05-08T06:37:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccf7fe640 1 -- 172.21.15.163:0/772555495 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8cbc00c360 con 0x7f8cbc003a50 2024-05-08T06:37:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8cc7fff640 1 --2- 172.21.15.163:0/772555495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cbc004800 0x7f8cbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8ccffff640 1 --2- 172.21.15.163:0/772555495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc003a50 0x7f8cbc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.550+0000 7f8cd486c640 1 --2- 172.21.15.163:0/772555495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cbc0051c0 0x7f8cbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.551+0000 7f8cc7fff640 1 -- 172.21.15.163:0/772555495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc003a50 msgr2=0x7f8cbc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.551+0000 7f8cc7fff640 1 --2- 172.21.15.163:0/772555495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc003a50 0x7f8cbc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.551+0000 7f8cc7fff640 1 -- 172.21.15.163:0/772555495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cbc0051c0 msgr2=0x7f8cbc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.551+0000 7f8cc7fff640 1 --2- 172.21.15.163:0/772555495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cbc0051c0 0x7f8cbc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.551+0000 7f8cd486c640 1 -- 172.21.15.163:0/772555495 reap_dead start 2024-05-08T06:37:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.551+0000 7f8cc7fff640 1 -- 172.21.15.163:0/772555495 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cbc003520 con 0x7f8cbc004800 2024-05-08T06:37:11.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.551+0000 7f8cc7fff640 1 --2- 172.21.15.163:0/772555495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cbc004800 0x7f8cbc004c80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8001050 tx=0x7f8cbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25009aea0175bf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:11.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.551+0000 7f8ccf7fe640 1 -- 172.21.15.163:0/772555495 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8007d10 con 0x7f8cbc004800 2024-05-08T06:37:11.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.551+0000 7f8ccf7fe640 1 -- 172.21.15.163:0/772555495 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb8005ad0 con 0x7f8cbc004800 2024-05-08T06:37:11.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.551+0000 7f8ccf7fe640 1 -- 172.21.15.163:0/772555495 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8007070 con 0x7f8cbc004800 2024-05-08T06:37:11.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.552+0000 7f8cd62cf640 1 -- 172.21.15.163:0/772555495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cbc004800 msgr2=0x7f8cbc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.552+0000 7f8cd62cf640 1 --2- 172.21.15.163:0/772555495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cbc004800 0x7f8cbc004c80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8001050 tx=0x7f8cbc005830 comp rx=0 tx=0).stop 2024-05-08T06:37:11.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.552+0000 7f8cd62cf640 1 -- 172.21.15.163:0/772555495 shutdown_connections 2024-05-08T06:37:11.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.552+0000 7f8cd62cf640 1 --2- 172.21.15.163:0/772555495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cbc004800 0x7f8cbc004c80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.552+0000 7f8cd62cf640 1 -- 172.21.15.163:0/772555495 >> 172.21.15.163:0/772555495 conn(0x7f8cd00682f0 msgr2=0x7f8cd0068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:11.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.552+0000 7f8cd62cf640 1 -- 172.21.15.163:0/772555495 shutdown_connections 2024-05-08T06:37:11.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.552+0000 7f8cd62cf640 1 -- 172.21.15.163:0/772555495 wait complete. 2024-05-08T06:37:11.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.553+0000 7f8cd62cf640 1 Processor -- start 2024-05-08T06:37:11.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.553+0000 7f8cd62cf640 1 -- start start 2024-05-08T06:37:11.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cd62cf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc004800 0x7f8cd0159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cd62cf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cd00b4d30 0x7f8cd0161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cd62cf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cd0158730 0x7f8cd0158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cd62cf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd0163220 con 0x7f8cd00b4d30 2024-05-08T06:37:11.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cd62cf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd01630a0 con 0x7f8cd0158730 2024-05-08T06:37:11.558 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cd62cf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd015f900 con 0x7f8cbc004800 2024-05-08T06:37:11.558 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cc7fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cd00b4d30 0x7f8cd0161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.558 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cc7fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cd00b4d30 0x7f8cd0161cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40542/0 (socket says 172.21.15.163:40542) 2024-05-08T06:37:11.558 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cc7fff640 1 -- 172.21.15.163:0/385229478 learned_addr learned my addr 172.21.15.163:0/385229478 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:11.558 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8ccffff640 1 --2- 172.21.15.163:0/385229478 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc004800 0x7f8cd0159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.558 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cd486c640 1 --2- 172.21.15.163:0/385229478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cd0158730 0x7f8cd0158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.558 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cc7fff640 1 -- 172.21.15.163:0/385229478 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc004800 msgr2=0x7f8cd0159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.558 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cc7fff640 1 --2- 172.21.15.163:0/385229478 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc004800 0x7f8cd0159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.558 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cc7fff640 1 -- 172.21.15.163:0/385229478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cd0158730 msgr2=0x7f8cd0158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.558 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cc7fff640 1 --2- 172.21.15.163:0/385229478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cd0158730 0x7f8cd0158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.558 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cc7fff640 1 -- 172.21.15.163:0/385229478 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd015a290 con 0x7f8cd00b4d30 2024-05-08T06:37:11.558 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cc7fff640 1 --2- 172.21.15.163:0/385229478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cd00b4d30 0x7f8cd0161cf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8001050 tx=0x7f8cd015fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:11.558 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8ccd7fa640 1 -- 172.21.15.163:0/385229478 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8041020 con 0x7f8cd00b4d30 2024-05-08T06:37:11.558 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8ccd7fa640 1 -- 172.21.15.163:0/385229478 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb8038040 con 0x7f8cd00b4d30 2024-05-08T06:37:11.559 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8ccd7fa640 1 -- 172.21.15.163:0/385229478 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8007070 con 0x7f8cd00b4d30 2024-05-08T06:37:11.559 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cd62cf640 1 -- 172.21.15.163:0/385229478 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd01590f0 con 0x7f8cd00b4d30 2024-05-08T06:37:11.559 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.554+0000 7f8cd62cf640 1 -- 172.21.15.163:0/385229478 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd015a810 con 0x7f8cd00b4d30 2024-05-08T06:37:11.559 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.556+0000 7f8cd62cf640 1 -- 172.21.15.163:0/385229478 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd006a550 con 0x7f8cd00b4d30 2024-05-08T06:37:11.559 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.556+0000 7f8ccd7fa640 1 -- 172.21.15.163:0/385229478 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8cb8044020 con 0x7f8cd00b4d30 2024-05-08T06:37:11.559 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.557+0000 7f8ccd7fa640 1 --2- 172.21.15.163:0/385229478 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8cb0076300 0x7f8cb00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.559 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.557+0000 7f8ccd7fa640 1 -- 172.21.15.163:0/385229478 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8cb80b7740 con 0x7f8cd00b4d30 2024-05-08T06:37:11.559 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.557+0000 7f8ccffff640 1 --2- 172.21.15.163:0/385229478 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8cb0076300 0x7f8cb00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.559 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.557+0000 7f8ccffff640 1 --2- 172.21.15.163:0/385229478 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8cb0076300 0x7f8cb00787c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00017a0 tx=0x7f8cc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:11.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.561+0000 7f8ccd7fa640 1 -- 172.21.15.163:0/385229478 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8cb8005aa0 con 0x7f8cd00b4d30 2024-05-08T06:37:11.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.779+0000 7f8cd62cf640 1 -- 172.21.15.163:0/385229478 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cd0072750 con 0x7f8cd00b4d30 2024-05-08T06:37:11.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.780+0000 7f8ccd7fa640 1 -- 172.21.15.163:0/385229478 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f8cb8089090 con 0x7f8cd00b4d30 2024-05-08T06:37:11.779 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:11.779 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2413407842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2413407842},{"type":"v1","addr":"172.21.15.163:6835","nonce":2413407842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2405203133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2405203133},{"type":"v1","addr":"172.21.15.163:6837","nonce":2405203133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3301087897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3301087897},{"type":"v1","addr":"172.21.15.192:6833","nonce":3301087897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:36:40.985978+0000","modified":"2024-05-08T06:36:43.900778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/354782184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":354782184},{"type":"v1","addr":"172.21.15.192:6835","nonce":354782184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-05-08T06:37:11.779 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 7 2024-05-08T06:37:11.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.782+0000 7f8cd62cf640 1 -- 172.21.15.163:0/385229478 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8cb0076300 msgr2=0x7f8cb00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.782+0000 7f8cd62cf640 1 --2- 172.21.15.163:0/385229478 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8cb0076300 0x7f8cb00787c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00017a0 tx=0x7f8cc0009040 comp rx=0 tx=0).stop 2024-05-08T06:37:11.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.782+0000 7f8cd62cf640 1 -- 172.21.15.163:0/385229478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cd00b4d30 msgr2=0x7f8cd0161cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.782+0000 7f8cd62cf640 1 --2- 172.21.15.163:0/385229478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cd00b4d30 0x7f8cd0161cf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8001050 tx=0x7f8cd015fe70 comp rx=0 tx=0).stop 2024-05-08T06:37:11.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.782+0000 7f8cd62cf640 1 -- 172.21.15.163:0/385229478 shutdown_connections 2024-05-08T06:37:11.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.782+0000 7f8cd62cf640 1 --2- 172.21.15.163:0/385229478 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8cb0076300 0x7f8cb00787c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.782+0000 7f8cd62cf640 1 --2- 172.21.15.163:0/385229478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cd0158730 0x7f8cd0158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.782+0000 7f8cd62cf640 1 --2- 172.21.15.163:0/385229478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cd00b4d30 0x7f8cd0161cf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.782+0000 7f8cd62cf640 1 --2- 172.21.15.163:0/385229478 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc004800 0x7f8cd0159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.782+0000 7f8cd62cf640 1 -- 172.21.15.163:0/385229478 >> 172.21.15.163:0/385229478 conn(0x7f8cd00682f0 msgr2=0x7f8cd00ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:11.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.782+0000 7f8cd62cf640 1 -- 172.21.15.163:0/385229478 shutdown_connections 2024-05-08T06:37:11.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.782+0000 7f8cd62cf640 1 -- 172.21.15.163:0/385229478 wait complete. 2024-05-08T06:37:11.793 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:11.880 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T06:37:11.883 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T06:37:11.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.894+0000 7f8d45340640 1 Processor -- start 2024-05-08T06:37:11.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.895+0000 7f8d45340640 1 -- start start 2024-05-08T06:37:11.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.895+0000 7f8d45340640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8d40140070 0x7f8d40071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.895+0000 7f8d45340640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8d400722f0 0x7f8d40142810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.895+0000 7f8d45340640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40142d50 0x7f8d40160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.895+0000 7f8d45340640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f8d401431a0 con 0x7f8d400722f0 2024-05-08T06:37:11.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.895+0000 7f8d45340640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8d40072e50 con 0x7f8d40140070 2024-05-08T06:37:11.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.895+0000 7f8d45340640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8d40140450 con 0x7f8d40142d50 2024-05-08T06:37:11.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.895+0000 7f8d37fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8d400722f0 0x7f8d40142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.896+0000 7f8d37fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8d400722f0 0x7f8d40142810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40556/0 (socket says 172.21.15.163:40556) 2024-05-08T06:37:11.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.896+0000 7f8d37fff640 1 -- 172.21.15.163:0/4184029813 learned_addr learned my addr 172.21.15.163:0/4184029813 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:11.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.896+0000 7f8d3effd640 1 --2- 172.21.15.163:0/4184029813 >> v2:172.21.15.192:3300/0 conn(0x7f8d40140070 0x7f8d40071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.896+0000 7f8d3f7fe640 1 --2- 172.21.15.163:0/4184029813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40142d50 0x7f8d40160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.896+0000 7f8d37fff640 1 -- 172.21.15.163:0/4184029813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40142d50 msgr2=0x7f8d40160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:11.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.896+0000 7f8d37fff640 1 --2- 172.21.15.163:0/4184029813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40142d50 0x7f8d40160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.896+0000 7f8d37fff640 1 -- 172.21.15.163:0/4184029813 >> v2:172.21.15.192:3300/0 conn(0x7f8d40140070 msgr2=0x7f8d40071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:11.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.896+0000 7f8d37fff640 1 --2- 172.21.15.163:0/4184029813 >> v2:172.21.15.192:3300/0 conn(0x7f8d40140070 0x7f8d40071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.896+0000 7f8d37fff640 1 -- 172.21.15.163:0/4184029813 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d40072890 con 0x7f8d400722f0 2024-05-08T06:37:11.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.897+0000 7f8d37fff640 1 --2- 172.21.15.163:0/4184029813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d400722f0 0x7f8d40142810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8d400697c0 tx=0x7f8d2802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1055dcba28c54e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:11.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.897+0000 7f8d3e7fc640 1 -- 172.21.15.163:0/4184029813 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28037070 con 0x7f8d400722f0 2024-05-08T06:37:11.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.897+0000 7f8d3e7fc640 1 -- 172.21.15.163:0/4184029813 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d2802ae00 con 0x7f8d400722f0 2024-05-08T06:37:11.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.897+0000 7f8d3e7fc640 1 -- 172.21.15.163:0/4184029813 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28032660 con 0x7f8d400722f0 2024-05-08T06:37:11.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.898+0000 7f8d45340640 1 -- 172.21.15.163:0/4184029813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d400722f0 msgr2=0x7f8d40142810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.898+0000 7f8d45340640 1 --2- 172.21.15.163:0/4184029813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d400722f0 0x7f8d40142810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8d400697c0 tx=0x7f8d2802a0b0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.898+0000 7f8d45340640 1 -- 172.21.15.163:0/4184029813 shutdown_connections 2024-05-08T06:37:11.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.898+0000 7f8d45340640 1 --2- 172.21.15.163:0/4184029813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40142d50 0x7f8d40160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.898+0000 7f8d45340640 1 --2- 172.21.15.163:0/4184029813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d400722f0 0x7f8d40142810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.898+0000 7f8d45340640 1 --2- 172.21.15.163:0/4184029813 >> v2:172.21.15.192:3300/0 conn(0x7f8d40140070 0x7f8d40071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.898+0000 7f8d45340640 1 -- 172.21.15.163:0/4184029813 >> 172.21.15.163:0/4184029813 conn(0x7f8d400b4da0 msgr2=0x7f8d400b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:11.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.898+0000 7f8d45340640 1 -- 172.21.15.163:0/4184029813 shutdown_connections 2024-05-08T06:37:11.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.898+0000 7f8d45340640 1 -- 172.21.15.163:0/4184029813 wait complete. 2024-05-08T06:37:11.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.899+0000 7f8d45340640 1 Processor -- start 2024-05-08T06:37:11.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.899+0000 7f8d45340640 1 -- start start 2024-05-08T06:37:11.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d45340640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d400722f0 0x7f8d401605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d45340640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d40140070 0x7f8d401f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d45340640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40142d50 0x7f8d40143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d45340640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8d401435b0 con 0x7f8d400722f0 2024-05-08T06:37:11.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d45340640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8d401408b0 con 0x7f8d40140070 2024-05-08T06:37:11.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d45340640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8d40161220 con 0x7f8d40142d50 2024-05-08T06:37:11.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d3effd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d400722f0 0x7f8d401605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d3effd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d400722f0 0x7f8d401605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40558/0 (socket says 172.21.15.163:40558) 2024-05-08T06:37:11.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d3effd640 1 -- 172.21.15.163:0/3416852735 learned_addr learned my addr 172.21.15.163:0/3416852735 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:11.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d3f7fe640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40142d50 0x7f8d40143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d37fff640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d40140070 0x7f8d401f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d3effd640 1 -- 172.21.15.163:0/3416852735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40142d50 msgr2=0x7f8d40143730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d3effd640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40142d50 0x7f8d40143730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d3effd640 1 -- 172.21.15.163:0/3416852735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d40140070 msgr2=0x7f8d401f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:11.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d3effd640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d40140070 0x7f8d401f2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:11.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d3effd640 1 -- 172.21.15.163:0/3416852735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4013e630 con 0x7f8d400722f0 2024-05-08T06:37:11.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d37fff640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d40140070 0x7f8d401f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:11.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d3effd640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d400722f0 0x7f8d401605c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000e8c0 tx=0x7f8d3000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:11.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d377fe640 1 -- 172.21.15.163:0/3416852735 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30017070 con 0x7f8d400722f0 2024-05-08T06:37:11.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d377fe640 1 -- 172.21.15.163:0/3416852735 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d3000c940 con 0x7f8d400722f0 2024-05-08T06:37:11.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d45340640 1 -- 172.21.15.163:0/3416852735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4013e910 con 0x7f8d400722f0 2024-05-08T06:37:11.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d377fe640 1 -- 172.21.15.163:0/3416852735 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3000cd10 con 0x7f8d400722f0 2024-05-08T06:37:11.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.900+0000 7f8d45340640 1 -- 172.21.15.163:0/3416852735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4013eed0 con 0x7f8d400722f0 2024-05-08T06:37:11.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.901+0000 7f8d45340640 1 -- 172.21.15.163:0/3416852735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d4006a550 con 0x7f8d400722f0 2024-05-08T06:37:11.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.902+0000 7f8d377fe640 1 -- 172.21.15.163:0/3416852735 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8d3001a070 con 0x7f8d400722f0 2024-05-08T06:37:11.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.902+0000 7f8d377fe640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8d200762d0 0x7f8d20078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:11.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.903+0000 7f8d377fe640 1 -- 172.21.15.163:0/3416852735 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8d300978a0 con 0x7f8d400722f0 2024-05-08T06:37:11.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.903+0000 7f8d37fff640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8d200762d0 0x7f8d20078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:11.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.903+0000 7f8d37fff640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8d200762d0 0x7f8d20078790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8d28034000 tx=0x7f8d2802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:11.906 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:11.907+0000 7f8d377fe640 1 -- 172.21.15.163:0/3416852735 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8d30060bb0 con 0x7f8d400722f0 2024-05-08T06:37:12.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.127+0000 7f8d45340640 1 -- 172.21.15.163:0/3416852735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d40144490 con 0x7f8d400722f0 2024-05-08T06:37:12.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.127+0000 7f8d377fe640 1 -- 172.21.15.163:0/3416852735 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f8d30064eb0 con 0x7f8d400722f0 2024-05-08T06:37:12.127 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:12.127 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2413407842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2413407842},{"type":"v1","addr":"172.21.15.163:6835","nonce":2413407842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2405203133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2405203133},{"type":"v1","addr":"172.21.15.163:6837","nonce":2405203133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3301087897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3301087897},{"type":"v1","addr":"172.21.15.192:6833","nonce":3301087897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:36:40.985978+0000","modified":"2024-05-08T06:36:43.900778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/354782184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":354782184},{"type":"v1","addr":"172.21.15.192:6835","nonce":354782184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-05-08T06:37:12.127 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 7 2024-05-08T06:37:12.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.129+0000 7f8d45340640 1 -- 172.21.15.163:0/3416852735 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8d200762d0 msgr2=0x7f8d20078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.129+0000 7f8d45340640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8d200762d0 0x7f8d20078790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8d28034000 tx=0x7f8d2802e000 comp rx=0 tx=0).stop 2024-05-08T06:37:12.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.129+0000 7f8d45340640 1 -- 172.21.15.163:0/3416852735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d400722f0 msgr2=0x7f8d401605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.129+0000 7f8d45340640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d400722f0 0x7f8d401605c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000e8c0 tx=0x7f8d3000ed80 comp rx=0 tx=0).stop 2024-05-08T06:37:12.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.129+0000 7f8d45340640 1 -- 172.21.15.163:0/3416852735 shutdown_connections 2024-05-08T06:37:12.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.129+0000 7f8d45340640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8d200762d0 0x7f8d20078790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.129+0000 7f8d45340640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d40142d50 0x7f8d40143730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.129+0000 7f8d45340640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d40140070 0x7f8d401f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.129+0000 7f8d45340640 1 --2- 172.21.15.163:0/3416852735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d400722f0 0x7f8d401605c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.129+0000 7f8d45340640 1 -- 172.21.15.163:0/3416852735 >> 172.21.15.163:0/3416852735 conn(0x7f8d400b4da0 msgr2=0x7f8d400d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:12.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.129+0000 7f8d45340640 1 -- 172.21.15.163:0/3416852735 shutdown_connections 2024-05-08T06:37:12.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.129+0000 7f8d45340640 1 -- 172.21.15.163:0/3416852735 wait complete. 2024-05-08T06:37:12.140 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:37:12.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.241+0000 7f5e3ab32640 1 Processor -- start 2024-05-08T06:37:12.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.241+0000 7f5e3ab32640 1 -- start start 2024-05-08T06:37:12.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e3ab32640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e340af310 0x7f5e340d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e3ab32640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e34074830 con 0x7f5e340ae180 2024-05-08T06:37:12.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e3ab32640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e34073f90 con 0x7f5e3415d130 2024-05-08T06:37:12.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e3ab32640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5e34074270 con 0x7f5e340af310 2024-05-08T06:37:12.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e390cf640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5e340ae180 0x7f5e3406edc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60574/0 (socket says 172.21.15.163:60574) 2024-05-08T06:37:12.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e390cf640 1 -- 172.21.15.163:0/1949868573 learned_addr learned my addr 172.21.15.163:0/1949868573 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:12.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2bfff640 1 --2- 172.21.15.163:0/1949868573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e340af310 0x7f5e340d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1420721886 0 0) 0x7f5e34074830 con 0x7f5e340ae180 2024-05-08T06:37:12.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e1c003520 con 0x7f5e340ae180 2024-05-08T06:37:12.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1718453297 0 0) 0x7f5e34073f90 con 0x7f5e3415d130 2024-05-08T06:37:12.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e34074830 con 0x7f5e3415d130 2024-05-08T06:37:12.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2653704966 0 0) 0x7f5e1c003520 con 0x7f5e340ae180 2024-05-08T06:37:12.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e340af310 msgr2=0x7f5e340d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:12.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2b7fe640 1 --2- 172.21.15.163:0/1949868573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e340af310 0x7f5e340d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 >> v1:172.21.15.192:6789/0 conn(0x7f5e3415d130 legacy=0x7f5e340aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e34074550 con 0x7f5e340ae180 2024-05-08T06:37:12.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f5e2c001a70 con 0x7f5e340ae180 2024-05-08T06:37:12.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2bfff640 1 --2- 172.21.15.163:0/1949868573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e340af310 0x7f5e340d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:12.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 >> v1:172.21.15.163:6789/0 conn(0x7f5e340ae180 legacy=0x7f5e3406edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2b7fe640 1 --2- 172.21.15.163:0/1949868573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e1c003c10 0x7f5e1c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2b7fe640 1 --2- 172.21.15.163:0/1949868573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e340af310 0x7f5e1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.242+0000 7f5e2b7fe640 1 --2- 172.21.15.163:0/1949868573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e1c004d40 0x7f5e1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.243+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5e34074270 con 0x7f5e1c004d40 2024-05-08T06:37:12.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.243+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c00bb20 con 0x7f5e1c003c10 2024-05-08T06:37:12.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.243+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5e1c00bd20 con 0x7f5e340af310 2024-05-08T06:37:12.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.243+0000 7f5e390cf640 1 --2- 172.21.15.163:0/1949868573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e1c004d40 0x7f5e1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.243+0000 7f5e388ce640 1 --2- 172.21.15.163:0/1949868573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e1c003c10 0x7f5e1c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.243+0000 7f5e2bfff640 1 --2- 172.21.15.163:0/1949868573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e340af310 0x7f5e1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.243+0000 7f5e390cf640 1 -- 172.21.15.163:0/1949868573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e340af310 msgr2=0x7f5e1c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:12.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.243+0000 7f5e390cf640 1 --2- 172.21.15.163:0/1949868573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e340af310 0x7f5e1c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.243+0000 7f5e390cf640 1 -- 172.21.15.163:0/1949868573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e1c003c10 msgr2=0x7f5e1c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:12.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.243+0000 7f5e390cf640 1 --2- 172.21.15.163:0/1949868573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e1c003c10 0x7f5e1c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.243+0000 7f5e390cf640 1 -- 172.21.15.163:0/1949868573 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1c003520 con 0x7f5e1c004d40 2024-05-08T06:37:12.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.244+0000 7f5e390cf640 1 --2- 172.21.15.163:0/1949868573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e1c004d40 0x7f5e1c0051c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00a7f0 tx=0x7f5e1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942c054f5b739b03 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:12.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.244+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00a410 con 0x7f5e1c004d40 2024-05-08T06:37:12.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.244+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e2c02f070 con 0x7f5e1c004d40 2024-05-08T06:37:12.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.244+0000 7f5e2b7fe640 1 -- 172.21.15.163:0/1949868573 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c02bb80 con 0x7f5e1c004d40 2024-05-08T06:37:12.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.244+0000 7f5e3ab32640 1 -- 172.21.15.163:0/1949868573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e1c004d40 msgr2=0x7f5e1c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.244+0000 7f5e3ab32640 1 --2- 172.21.15.163:0/1949868573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e1c004d40 0x7f5e1c0051c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00a7f0 tx=0x7f5e1c005830 comp rx=0 tx=0).stop 2024-05-08T06:37:12.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.244+0000 7f5e390cf640 1 -- 172.21.15.163:0/1949868573 reap_dead start 2024-05-08T06:37:12.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.244+0000 7f5e3ab32640 1 -- 172.21.15.163:0/1949868573 shutdown_connections 2024-05-08T06:37:12.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.244+0000 7f5e3ab32640 1 -- 172.21.15.163:0/1949868573 >> 172.21.15.163:0/1949868573 conn(0x7f5e3406c500 msgr2=0x7f5e3406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:12.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.244+0000 7f5e3ab32640 1 -- 172.21.15.163:0/1949868573 shutdown_connections 2024-05-08T06:37:12.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.244+0000 7f5e3ab32640 1 -- 172.21.15.163:0/1949868573 wait complete. 2024-05-08T06:37:12.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.245+0000 7f5e3ab32640 1 Processor -- start 2024-05-08T06:37:12.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.246+0000 7f5e3ab32640 1 -- start start 2024-05-08T06:37:12.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.246+0000 7f5e3ab32640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e34154610 0x7f5e3409d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.246+0000 7f5e3ab32640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e341549f0 0x7f5e3406ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.246+0000 7f5e3ab32640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e341f74e0 0x7f5e3406f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.246+0000 7f5e3ab32640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5e340d08a0 con 0x7f5e34154610 2024-05-08T06:37:12.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.246+0000 7f5e3ab32640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5e340d0720 con 0x7f5e341f74e0 2024-05-08T06:37:12.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.246+0000 7f5e3ab32640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5e340d0a20 con 0x7f5e341549f0 2024-05-08T06:37:12.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.246+0000 7f5e388ce640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e34154610 0x7f5e3409d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e388ce640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e34154610 0x7f5e3409d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40580/0 (socket says 172.21.15.163:40580) 2024-05-08T06:37:12.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e388ce640 1 -- 172.21.15.163:0/2629911454 learned_addr learned my addr 172.21.15.163:0/2629911454 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:12.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e390cf640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e341f74e0 0x7f5e3406f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e2bfff640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e341549f0 0x7f5e3406ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e388ce640 1 -- 172.21.15.163:0/2629911454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e341549f0 msgr2=0x7f5e3406ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e388ce640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e341549f0 0x7f5e3406ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e388ce640 1 -- 172.21.15.163:0/2629911454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e341f74e0 msgr2=0x7f5e3406f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e388ce640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e341f74e0 0x7f5e3406f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e388ce640 1 -- 172.21.15.163:0/2629911454 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3409b3c0 con 0x7f5e34154610 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e2bfff640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e341549f0 0x7f5e3406ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e388ce640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e34154610 0x7f5e3409d790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000f880 tx=0x7f5e3000fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e297fa640 1 -- 172.21.15.163:0/2629911454 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e300052e0 con 0x7f5e34154610 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e297fa640 1 -- 172.21.15.163:0/2629911454 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e30018070 con 0x7f5e34154610 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e3ab32640 1 -- 172.21.15.163:0/2629911454 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3409dfe0 con 0x7f5e34154610 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e297fa640 1 -- 172.21.15.163:0/2629911454 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3000d970 con 0x7f5e34154610 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.247+0000 7f5e3ab32640 1 -- 172.21.15.163:0/2629911454 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e3409dcd0 con 0x7f5e34154610 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.248+0000 7f5e3ab32640 1 -- 172.21.15.163:0/2629911454 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e3406a550 con 0x7f5e34154610 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.249+0000 7f5e297fa640 1 -- 172.21.15.163:0/2629911454 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5e3000db10 con 0x7f5e34154610 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.249+0000 7f5e297fa640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5e20076300 0x7f5e200787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.249+0000 7f5e2bfff640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5e20076300 0x7f5e200787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.252 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.249+0000 7f5e297fa640 1 -- 172.21.15.163:0/2629911454 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5e300973d0 con 0x7f5e34154610 2024-05-08T06:37:12.252 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.250+0000 7f5e2bfff640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5e20076300 0x7f5e200787c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5e341e8460 tx=0x7f5e24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:12.252 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.253+0000 7f5e297fa640 1 -- 172.21.15.163:0/2629911454 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5e30060dc0 con 0x7f5e34154610 2024-05-08T06:37:12.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.437+0000 7f5e3ab32640 1 -- 172.21.15.163:0/2629911454 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e34072db0 con 0x7f5e34154610 2024-05-08T06:37:12.437 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.438+0000 7f5e297fa640 1 -- 172.21.15.163:0/2629911454 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f5e300606c0 con 0x7f5e34154610 2024-05-08T06:37:12.438 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:12.438 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:36:42.641260+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:36:38.853461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:36:39.952926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:37:12.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.440+0000 7f5e3ab32640 1 -- 172.21.15.163:0/2629911454 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5e20076300 msgr2=0x7f5e200787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.440+0000 7f5e3ab32640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5e20076300 0x7f5e200787c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5e341e8460 tx=0x7f5e24008040 comp rx=0 tx=0).stop 2024-05-08T06:37:12.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.441+0000 7f5e3ab32640 1 -- 172.21.15.163:0/2629911454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e34154610 msgr2=0x7f5e3409d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.441+0000 7f5e3ab32640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e34154610 0x7f5e3409d790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000f880 tx=0x7f5e3000fc80 comp rx=0 tx=0).stop 2024-05-08T06:37:12.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.441+0000 7f5e3ab32640 1 -- 172.21.15.163:0/2629911454 shutdown_connections 2024-05-08T06:37:12.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.441+0000 7f5e3ab32640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f5e20076300 0x7f5e200787c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.441+0000 7f5e3ab32640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e341f74e0 0x7f5e3406f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.441+0000 7f5e3ab32640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e341549f0 0x7f5e3406ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.441+0000 7f5e3ab32640 1 --2- 172.21.15.163:0/2629911454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e34154610 0x7f5e3409d790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.440 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.441+0000 7f5e3ab32640 1 -- 172.21.15.163:0/2629911454 >> 172.21.15.163:0/2629911454 conn(0x7f5e3406c500 msgr2=0x7f5e340cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:12.441 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.441+0000 7f5e3ab32640 1 -- 172.21.15.163:0/2629911454 shutdown_connections 2024-05-08T06:37:12.441 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.441+0000 7f5e3ab32640 1 -- 172.21.15.163:0/2629911454 wait complete. 2024-05-08T06:37:12.451 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T06:37:12.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.553+0000 7f4ecff96640 1 Processor -- start 2024-05-08T06:37:12.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.553+0000 7f4ecff96640 1 -- start start 2024-05-08T06:37:12.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.554+0000 7f4ecff96640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec80d3d10 0x7f4ec81629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.554+0000 7f4ecff96640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4ec8162f00 0x7f4ec806d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.554+0000 7f4ecff96640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec80d1560 con 0x7f4ec80d3930 2024-05-08T06:37:12.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.554+0000 7f4ecff96640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec8163390 con 0x7f4ec8162f00 2024-05-08T06:37:12.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.554+0000 7f4ecff96640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec80d0cc0 con 0x7f4ec80d3d10 2024-05-08T06:37:12.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.554+0000 7f4ecd531640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec80d3d10 0x7f4ec81629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.555+0000 7f4ece533640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4ec8162f00 0x7f4ec806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.555+0000 7f4ecdd32640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f4ec80d3930 0x7f4ec80d59a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:44238/0 (socket says 172.21.15.163:44238) 2024-05-08T06:37:12.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.555+0000 7f4ece533640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4ec8162f00 0x7f4ec806d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:49266/0 (socket says 172.21.15.163:49266) 2024-05-08T06:37:12.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.555+0000 7f4ecd531640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec80d3d10 0x7f4ec81629c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:58588/0 (socket says 172.21.15.163:58588) 2024-05-08T06:37:12.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.555+0000 7f4ecdd32640 1 -- 172.21.15.163:0/57733481 learned_addr learned my addr 172.21.15.163:0/57733481 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:12.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.555+0000 7f4ece533640 1 -- 172.21.15.163:0/57733481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec80d3d10 msgr2=0x7f4ec81629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:12.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.555+0000 7f4ece533640 1 --2- 172.21.15.163:0/57733481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec80d3d10 0x7f4ec81629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.555+0000 7f4ece533640 1 -- 172.21.15.163:0/57733481 >> v1:172.21.15.192:6789/0 conn(0x7f4ec80d3930 legacy=0x7f4ec80d59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.555+0000 7f4ece533640 1 -- 172.21.15.163:0/57733481 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec80d0fa0 con 0x7f4ec8162f00 2024-05-08T06:37:12.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.555+0000 7f4eccd30640 1 -- 172.21.15.163:0/57733481 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3953966398 0 0) 0x7f4ec80d1560 con 0x7f4ec80d3930 2024-05-08T06:37:12.555 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.555+0000 7f4ecd531640 1 --2- 172.21.15.163:0/57733481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec80d3d10 0x7f4ec81629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:12.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.556+0000 7f4ece533640 1 --2- 172.21.15.163:0/57733481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec8162f00 0x7f4ec806d960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009d90 tx=0x7f4ec002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8ccfd50d0e4772a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:12.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.556+0000 7f4eccd30640 1 -- 172.21.15.163:0/57733481 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0007740 con 0x7f4ec8162f00 2024-05-08T06:37:12.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.556+0000 7f4eccd30640 1 -- 172.21.15.163:0/57733481 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec00078e0 con 0x7f4ec8162f00 2024-05-08T06:37:12.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.556+0000 7f4eccd30640 1 -- 172.21.15.163:0/57733481 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0007cb0 con 0x7f4ec8162f00 2024-05-08T06:37:12.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.556+0000 7f4ecff96640 1 -- 172.21.15.163:0/57733481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec8162f00 msgr2=0x7f4ec806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.556+0000 7f4ecff96640 1 --2- 172.21.15.163:0/57733481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec8162f00 0x7f4ec806d960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009d90 tx=0x7f4ec002a660 comp rx=0 tx=0).stop 2024-05-08T06:37:12.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.556+0000 7f4ecff96640 1 -- 172.21.15.163:0/57733481 shutdown_connections 2024-05-08T06:37:12.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.556+0000 7f4ecff96640 1 --2- 172.21.15.163:0/57733481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec8162f00 0x7f4ec806d960 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.556+0000 7f4ecff96640 1 --2- 172.21.15.163:0/57733481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec80d3d10 0x7f4ec81629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.556+0000 7f4ecff96640 1 -- 172.21.15.163:0/57733481 >> 172.21.15.163:0/57733481 conn(0x7f4ec808cfa0 msgr2=0x7f4ec80d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:12.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.557+0000 7f4ecff96640 1 -- 172.21.15.163:0/57733481 shutdown_connections 2024-05-08T06:37:12.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.557+0000 7f4ecff96640 1 -- 172.21.15.163:0/57733481 wait complete. 2024-05-08T06:37:12.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.558+0000 7f4ecff96640 1 Processor -- start 2024-05-08T06:37:12.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.558+0000 7f4ecff96640 1 -- start start 2024-05-08T06:37:12.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.558+0000 7f4ecff96640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec80d3930 0x7f4ec809f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.558+0000 7f4ecff96640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80d3d10 0x7f4ec80cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecff96640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec8162f00 0x7f4ec815c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecff96640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec80d0970 con 0x7f4ec80d3d10 2024-05-08T06:37:12.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecff96640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec80d60b0 con 0x7f4ec8162f00 2024-05-08T06:37:12.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecff96640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec806dea0 con 0x7f4ec80d3930 2024-05-08T06:37:12.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecd531640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80d3d10 0x7f4ec80cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecd531640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80d3d10 0x7f4ec80cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40590/0 (socket says 172.21.15.163:40590) 2024-05-08T06:37:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecd531640 1 -- 172.21.15.163:0/3676487560 learned_addr learned my addr 172.21.15.163:0/3676487560 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecdd32640 1 --2- 172.21.15.163:0/3676487560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec80d3930 0x7f4ec809f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ece533640 1 --2- 172.21.15.163:0/3676487560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec8162f00 0x7f4ec815c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecd531640 1 -- 172.21.15.163:0/3676487560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec80d3930 msgr2=0x7f4ec809f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecd531640 1 --2- 172.21.15.163:0/3676487560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec80d3930 0x7f4ec809f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecd531640 1 -- 172.21.15.163:0/3676487560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec8162f00 msgr2=0x7f4ec815c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecd531640 1 --2- 172.21.15.163:0/3676487560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec8162f00 0x7f4ec815c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecd531640 1 -- 172.21.15.163:0/3676487560 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec80a2de0 con 0x7f4ec80d3d10 2024-05-08T06:37:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecd531640 1 --2- 172.21.15.163:0/3676487560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80d3d10 0x7f4ec80cdac0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800ebe0 tx=0x7f4eb800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ebe7fc640 1 -- 172.21.15.163:0/3676487560 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8017070 con 0x7f4ec80d3d10 2024-05-08T06:37:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ebe7fc640 1 -- 172.21.15.163:0/3676487560 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb800cc30 con 0x7f4ec80d3d10 2024-05-08T06:37:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecff96640 1 -- 172.21.15.163:0/3676487560 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec80a2540 con 0x7f4ec80d3d10 2024-05-08T06:37:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ebe7fc640 1 -- 172.21.15.163:0/3676487560 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8013470 con 0x7f4ec80d3d10 2024-05-08T06:37:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.559+0000 7f4ecff96640 1 -- 172.21.15.163:0/3676487560 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec80c8ed0 con 0x7f4ec80d3d10 2024-05-08T06:37:12.562 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.560+0000 7f4ecff96640 1 -- 172.21.15.163:0/3676487560 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec806a4b0 con 0x7f4ec80d3d10 2024-05-08T06:37:12.562 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.561+0000 7f4ebe7fc640 1 -- 172.21.15.163:0/3676487560 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4eb801a070 con 0x7f4ec80d3d10 2024-05-08T06:37:12.562 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.562+0000 7f4ebe7fc640 1 --2- 172.21.15.163:0/3676487560 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4eac0762d0 0x7f4eac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.562 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.562+0000 7f4ecdd32640 1 --2- 172.21.15.163:0/3676487560 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4eac0762d0 0x7f4eac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.562 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.562+0000 7f4ebe7fc640 1 -- 172.21.15.163:0/3676487560 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4eb8097440 con 0x7f4ec80d3d10 2024-05-08T06:37:12.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.562+0000 7f4ecdd32640 1 --2- 172.21.15.163:0/3676487560 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4eac0762d0 0x7f4eac078790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4000f60 tx=0x7f4ec4007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:12.565 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.566+0000 7f4ebe7fc640 1 -- 172.21.15.163:0/3676487560 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4eb8060760 con 0x7f4ec80d3d10 2024-05-08T06:37:12.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.790+0000 7f4ecff96640 1 -- 172.21.15.163:0/3676487560 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4ec80d3440 con 0x7f4ec80d3d10 2024-05-08T06:37:12.789 INFO:tasks.ceph.mon.a.smithi163.stderr:2024-05-08T06:37:12.791+0000 7fa54ba64640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:37:12.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.792+0000 7f4ebe7fc640 1 -- 172.21.15.163:0/3676487560 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7f4eb8064a60 con 0x7f4ec80d3d10 2024-05-08T06:37:12.791 INFO:teuthology.orchestra.run.smithi163.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T06:37:12.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.795+0000 7f4ecff96640 1 -- 172.21.15.163:0/3676487560 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4eac0762d0 msgr2=0x7f4eac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.795+0000 7f4ecff96640 1 --2- 172.21.15.163:0/3676487560 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4eac0762d0 0x7f4eac078790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4000f60 tx=0x7f4ec4007cf0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.795+0000 7f4ecff96640 1 -- 172.21.15.163:0/3676487560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80d3d10 msgr2=0x7f4ec80cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.795+0000 7f4ecff96640 1 --2- 172.21.15.163:0/3676487560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80d3d10 0x7f4ec80cdac0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800ebe0 tx=0x7f4eb800c520 comp rx=0 tx=0).stop 2024-05-08T06:37:12.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.795+0000 7f4ecff96640 1 -- 172.21.15.163:0/3676487560 shutdown_connections 2024-05-08T06:37:12.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.795+0000 7f4ecff96640 1 --2- 172.21.15.163:0/3676487560 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4eac0762d0 0x7f4eac078790 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.795+0000 7f4ecff96640 1 --2- 172.21.15.163:0/3676487560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec8162f00 0x7f4ec815c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.795+0000 7f4ecff96640 1 --2- 172.21.15.163:0/3676487560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80d3d10 0x7f4ec80cdac0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.795+0000 7f4ecff96640 1 --2- 172.21.15.163:0/3676487560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec80d3930 0x7f4ec809f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.795+0000 7f4ecff96640 1 -- 172.21.15.163:0/3676487560 >> 172.21.15.163:0/3676487560 conn(0x7f4ec808cfa0 msgr2=0x7f4ec8161520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:12.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.795+0000 7f4ecff96640 1 -- 172.21.15.163:0/3676487560 shutdown_connections 2024-05-08T06:37:12.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.795+0000 7f4ecff96640 1 -- 172.21.15.163:0/3676487560 wait complete. 2024-05-08T06:37:12.798 INFO:tasks.ceph.mds.d.smithi192.stderr: -6701> 2024-05-08T06:36:46.616+0000 7ff9e18cc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T06:37:12.805 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T06:37:12.831 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-08T06:37:12.910 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.911+0000 7f797e59e640 1 Processor -- start 2024-05-08T06:37:12.911 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.911+0000 7f797e59e640 1 -- start start 2024-05-08T06:37:12.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.912+0000 7f797e59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7978148e20 0x7f797806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.912+0000 7f797e59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7978162ea0 0x7f79780b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.912+0000 7f797e59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f797806de40 con 0x7f79780b6910 2024-05-08T06:37:12.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.912+0000 7f797e59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f797806e0e0 con 0x7f7978162ea0 2024-05-08T06:37:12.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.912+0000 7f797e59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f797806e3c0 con 0x7f7978148e20 2024-05-08T06:37:12.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.912+0000 7f7977fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7978148e20 0x7f797806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.912+0000 7f79777fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7978162ea0 0x7f79780b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.912+0000 7f797cb3b640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f79780b6910 0x7f7978071e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:44250/0 (socket says 172.21.15.163:44250) 2024-05-08T06:37:12.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.913+0000 7f7977fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7978148e20 0x7f797806d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:58604/0 (socket says 172.21.15.163:58604) 2024-05-08T06:37:12.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.913+0000 7f797cb3b640 1 -- 172.21.15.163:0/307692424 learned_addr learned my addr 172.21.15.163:0/307692424 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:12.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.913+0000 7f79777fe640 1 -- 172.21.15.163:0/307692424 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7978148e20 msgr2=0x7f797806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:12.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.913+0000 7f79777fe640 1 --2- 172.21.15.163:0/307692424 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7978148e20 0x7f797806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.913+0000 7f79777fe640 1 -- 172.21.15.163:0/307692424 >> v1:172.21.15.192:6789/0 conn(0x7f79780b6910 legacy=0x7f7978071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.913+0000 7f79777fe640 1 -- 172.21.15.163:0/307692424 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797806e6a0 con 0x7f7978162ea0 2024-05-08T06:37:12.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.913+0000 7f7977fff640 1 --2- 172.21.15.163:0/307692424 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7978148e20 0x7f797806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:12.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.914+0000 7f79777fe640 1 --2- 172.21.15.163:0/307692424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7978162ea0 0x7f79780b63d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f796c0097a0 tx=0x7f796c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=173d6bf5a600709a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:12.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.914+0000 7f7976ffd640 1 -- 172.21.15.163:0/307692424 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c037070 con 0x7f7978162ea0 2024-05-08T06:37:12.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.914+0000 7f7976ffd640 1 -- 172.21.15.163:0/307692424 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f796c032440 con 0x7f7978162ea0 2024-05-08T06:37:12.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.914+0000 7f7976ffd640 1 -- 172.21.15.163:0/307692424 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c032810 con 0x7f7978162ea0 2024-05-08T06:37:12.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.914+0000 7f797e59e640 1 -- 172.21.15.163:0/307692424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7978162ea0 msgr2=0x7f79780b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.914+0000 7f797e59e640 1 --2- 172.21.15.163:0/307692424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7978162ea0 0x7f79780b63d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f796c0097a0 tx=0x7f796c02a180 comp rx=0 tx=0).stop 2024-05-08T06:37:12.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.914+0000 7f797e59e640 1 -- 172.21.15.163:0/307692424 shutdown_connections 2024-05-08T06:37:12.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.914+0000 7f797e59e640 1 --2- 172.21.15.163:0/307692424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7978162ea0 0x7f79780b63d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.914+0000 7f797e59e640 1 --2- 172.21.15.163:0/307692424 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7978148e20 0x7f797806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.914+0000 7f797e59e640 1 -- 172.21.15.163:0/307692424 >> 172.21.15.163:0/307692424 conn(0x7f79780706c0 msgr2=0x7f7978070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:12.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.914+0000 7f797e59e640 1 -- 172.21.15.163:0/307692424 shutdown_connections 2024-05-08T06:37:12.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.914+0000 7f797e59e640 1 -- 172.21.15.163:0/307692424 wait complete. 2024-05-08T06:37:12.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.915+0000 7f797e59e640 1 Processor -- start 2024-05-08T06:37:12.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.916+0000 7f797e59e640 1 -- start start 2024-05-08T06:37:12.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.916+0000 7f797e59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79780b6910 0x7f797815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.916+0000 7f797e59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7978148e20 0x7f79781f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.916+0000 7f797e59e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978162ea0 0x7f7978147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.916+0000 7f797e59e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7978160e40 con 0x7f7978162ea0 2024-05-08T06:37:12.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.916+0000 7f797e59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7978160cc0 con 0x7f79780b6910 2024-05-08T06:37:12.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.916+0000 7f797e59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7978160fc0 con 0x7f7978148e20 2024-05-08T06:37:12.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f797cb3b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978162ea0 0x7f7978147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f79777fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7978148e20 0x7f79781f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f7977fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79780b6910 0x7f797815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f797cb3b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978162ea0 0x7f7978147cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40606/0 (socket says 172.21.15.163:40606) 2024-05-08T06:37:12.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f797cb3b640 1 -- 172.21.15.163:0/3425271350 learned_addr learned my addr 172.21.15.163:0/3425271350 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:12.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f7977fff640 1 -- 172.21.15.163:0/3425271350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7978148e20 msgr2=0x7f79781f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f7977fff640 1 --2- 172.21.15.163:0/3425271350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7978148e20 0x7f79781f4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f7977fff640 1 -- 172.21.15.163:0/3425271350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978162ea0 msgr2=0x7f7978147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:12.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f7977fff640 1 --2- 172.21.15.163:0/3425271350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978162ea0 0x7f7978147cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:12.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f7977fff640 1 -- 172.21.15.163:0/3425271350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7978144ea0 con 0x7f79780b6910 2024-05-08T06:37:12.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f7977fff640 1 --2- 172.21.15.163:0/3425271350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79780b6910 0x7f797815ef80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f796800e900 tx=0x7f796800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:12.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f7974ff9640 1 -- 172.21.15.163:0/3425271350 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968017070 con 0x7f79780b6910 2024-05-08T06:37:12.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f7974ff9640 1 -- 172.21.15.163:0/3425271350 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f796800ca00 con 0x7f79780b6910 2024-05-08T06:37:12.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f797e59e640 1 -- 172.21.15.163:0/3425271350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7978145180 con 0x7f79780b6910 2024-05-08T06:37:12.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f7974ff9640 1 -- 172.21.15.163:0/3425271350 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796800cdf0 con 0x7f79780b6910 2024-05-08T06:37:12.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.917+0000 7f797e59e640 1 -- 172.21.15.163:0/3425271350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7978146f80 con 0x7f79780b6910 2024-05-08T06:37:12.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.918+0000 7f797e59e640 1 -- 172.21.15.163:0/3425271350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797806a550 con 0x7f79780b6910 2024-05-08T06:37:12.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.919+0000 7f7974ff9640 1 -- 172.21.15.163:0/3425271350 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f796801a070 con 0x7f79780b6910 2024-05-08T06:37:12.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.919+0000 7f7974ff9640 1 --2- 172.21.15.163:0/3425271350 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f79640762d0 0x7f7964078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:12.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.919+0000 7f7974ff9640 1 -- 172.21.15.163:0/3425271350 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f79680977f0 con 0x7f79780b6910 2024-05-08T06:37:12.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.919+0000 7f79777fe640 1 --2- 172.21.15.163:0/3425271350 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f79640762d0 0x7f7964078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:12.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.920+0000 7f79777fe640 1 --2- 172.21.15.163:0/3425271350 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f79640762d0 0x7f7964078790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f796c034000 tx=0x7f796c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:12.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:12.924+0000 7f7974ff9640 1 -- 172.21.15.163:0/3425271350 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7968060aa0 con 0x7f79780b6910 2024-05-08T06:37:13.139 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.140+0000 7f797e59e640 1 -- 172.21.15.163:0/3425271350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f79780b61c0 con 0x7f79780b6910 2024-05-08T06:37:13.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.795+0000 7f7974ff9640 1 -- 172.21.15.163:0/3425271350 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f7968064da0 con 0x7f79780b6910 2024-05-08T06:37:13.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.797+0000 7f797e59e640 1 -- 172.21.15.163:0/3425271350 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f79640762d0 msgr2=0x7f7964078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:13.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.797+0000 7f797e59e640 1 --2- 172.21.15.163:0/3425271350 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f79640762d0 0x7f7964078790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f796c034000 tx=0x7f796c02e000 comp rx=0 tx=0).stop 2024-05-08T06:37:13.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.797+0000 7f797e59e640 1 -- 172.21.15.163:0/3425271350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79780b6910 msgr2=0x7f797815ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:13.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.797+0000 7f797e59e640 1 --2- 172.21.15.163:0/3425271350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79780b6910 0x7f797815ef80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f796800e900 tx=0x7f796800edc0 comp rx=0 tx=0).stop 2024-05-08T06:37:13.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.797+0000 7f797e59e640 1 -- 172.21.15.163:0/3425271350 shutdown_connections 2024-05-08T06:37:13.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.797+0000 7f797e59e640 1 --2- 172.21.15.163:0/3425271350 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f79640762d0 0x7f7964078790 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:13.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.797+0000 7f797e59e640 1 --2- 172.21.15.163:0/3425271350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7978162ea0 0x7f7978147cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:13.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.797+0000 7f797e59e640 1 --2- 172.21.15.163:0/3425271350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7978148e20 0x7f79781f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:13.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.797+0000 7f797e59e640 1 --2- 172.21.15.163:0/3425271350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79780b6910 0x7f797815ef80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:13.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.797+0000 7f797e59e640 1 -- 172.21.15.163:0/3425271350 >> 172.21.15.163:0/3425271350 conn(0x7f79780706c0 msgr2=0x7f7978071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:13.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.797+0000 7f797e59e640 1 -- 172.21.15.163:0/3425271350 shutdown_connections 2024-05-08T06:37:13.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.797+0000 7f797e59e640 1 -- 172.21.15.163:0/3425271350 wait complete. 2024-05-08T06:37:13.809 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T06:37:13.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.916+0000 7ff28dd5f640 1 Processor -- start 2024-05-08T06:37:13.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.916+0000 7ff28dd5f640 1 -- start start 2024-05-08T06:37:13.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff28dd5f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff28815c7d0 0x7ff28806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:13.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff28dd5f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff2880b49f0 0x7ff28815db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:13.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff28dd5f640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2880b4f80 con 0x7ff28806f010 2024-05-08T06:37:13.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff28dd5f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7ff2880b5260 con 0x7ff28815c7d0 2024-05-08T06:37:13.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff28dd5f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff28815d070 con 0x7ff2880b49f0 2024-05-08T06:37:13.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff2877fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff28815c7d0 0x7ff28806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:13.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff286ffd640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff28806f010 0x7ff2880b42e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60586/0 (socket says 172.21.15.163:60586) 2024-05-08T06:37:13.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff286ffd640 1 -- 172.21.15.163:0/774799451 learned_addr learned my addr 172.21.15.163:0/774799451 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:13.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff287fff640 1 --2- 172.21.15.163:0/774799451 >> v2:172.21.15.192:3300/0 conn(0x7ff2880b49f0 0x7ff28815db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:13.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff2877fe640 1 -- 172.21.15.163:0/774799451 >> v2:172.21.15.192:3300/0 conn(0x7ff2880b49f0 msgr2=0x7ff28815db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:13.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff2877fe640 1 --2- 172.21.15.163:0/774799451 >> v2:172.21.15.192:3300/0 conn(0x7ff2880b49f0 0x7ff28815db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:13.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff2877fe640 1 -- 172.21.15.163:0/774799451 >> v1:172.21.15.163:6789/0 conn(0x7ff28806f010 legacy=0x7ff2880b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:13.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff2877fe640 1 -- 172.21.15.163:0/774799451 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28815d350 con 0x7ff28815c7d0 2024-05-08T06:37:13.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff2867fc640 1 -- 172.21.15.163:0/774799451 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2352293889 0 0) 0x7ff2880b4f80 con 0x7ff28806f010 2024-05-08T06:37:13.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.917+0000 7ff287fff640 1 --2- 172.21.15.163:0/774799451 >> v2:172.21.15.192:3300/0 conn(0x7ff2880b49f0 0x7ff28815db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:13.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.918+0000 7ff2877fe640 1 --2- 172.21.15.163:0/774799451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28815c7d0 0x7ff28806ead0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff288074850 tx=0x7ff27802c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe97d9e14c98f6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:13.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.918+0000 7ff2867fc640 1 -- 172.21.15.163:0/774799451 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278032770 con 0x7ff28815c7d0 2024-05-08T06:37:13.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.918+0000 7ff2867fc640 1 -- 172.21.15.163:0/774799451 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff278038070 con 0x7ff28815c7d0 2024-05-08T06:37:13.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.918+0000 7ff2867fc640 1 -- 172.21.15.163:0/774799451 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278032b30 con 0x7ff28815c7d0 2024-05-08T06:37:13.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.919+0000 7ff28dd5f640 1 -- 172.21.15.163:0/774799451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28815c7d0 msgr2=0x7ff28806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:13.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.919+0000 7ff28dd5f640 1 --2- 172.21.15.163:0/774799451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28815c7d0 0x7ff28806ead0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff288074850 tx=0x7ff27802c870 comp rx=0 tx=0).stop 2024-05-08T06:37:13.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.919+0000 7ff28dd5f640 1 -- 172.21.15.163:0/774799451 shutdown_connections 2024-05-08T06:37:13.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.919+0000 7ff28dd5f640 1 --2- 172.21.15.163:0/774799451 >> v2:172.21.15.192:3300/0 conn(0x7ff2880b49f0 0x7ff28815db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:13.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.919+0000 7ff28dd5f640 1 --2- 172.21.15.163:0/774799451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28815c7d0 0x7ff28806ead0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:13.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.919+0000 7ff28dd5f640 1 -- 172.21.15.163:0/774799451 >> 172.21.15.163:0/774799451 conn(0x7ff2880b2b40 msgr2=0x7ff2880b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:13.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.919+0000 7ff28dd5f640 1 -- 172.21.15.163:0/774799451 shutdown_connections 2024-05-08T06:37:13.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.919+0000 7ff28dd5f640 1 -- 172.21.15.163:0/774799451 wait complete. 2024-05-08T06:37:13.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.920+0000 7ff28dd5f640 1 Processor -- start 2024-05-08T06:37:13.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.920+0000 7ff28dd5f640 1 -- start start 2024-05-08T06:37:13.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff28dd5f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff28806f010 0x7ff28814ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:13.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff28dd5f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2880b49f0 0x7ff2881f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:13.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff28dd5f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28815c7d0 0x7ff288154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:13.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff28dd5f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff28815cd30 con 0x7ff28815c7d0 2024-05-08T06:37:13.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff28dd5f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff28815cbb0 con 0x7ff2880b49f0 2024-05-08T06:37:13.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff28dd5f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff28806e240 con 0x7ff28806f010 2024-05-08T06:37:13.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff287fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28815c7d0 0x7ff288154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:13.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff287fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28815c7d0 0x7ff288154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40622/0 (socket says 172.21.15.163:40622) 2024-05-08T06:37:13.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff287fff640 1 -- 172.21.15.163:0/2166686373 learned_addr learned my addr 172.21.15.163:0/2166686373 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:13.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff286ffd640 1 --2- 172.21.15.163:0/2166686373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2880b49f0 0x7ff2881f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:13.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff2877fe640 1 --2- 172.21.15.163:0/2166686373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff28806f010 0x7ff28814ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:13.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff287fff640 1 -- 172.21.15.163:0/2166686373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff28806f010 msgr2=0x7ff28814ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:13.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff287fff640 1 --2- 172.21.15.163:0/2166686373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff28806f010 0x7ff28814ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:13.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff287fff640 1 -- 172.21.15.163:0/2166686373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2880b49f0 msgr2=0x7ff2881f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:13.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff287fff640 1 --2- 172.21.15.163:0/2166686373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2880b49f0 0x7ff2881f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:13.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.921+0000 7ff287fff640 1 -- 172.21.15.163:0/2166686373 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2880cf5e0 con 0x7ff28815c7d0 2024-05-08T06:37:13.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.922+0000 7ff287fff640 1 --2- 172.21.15.163:0/2166686373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28815c7d0 0x7ff288154e20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff28000e7e0 tx=0x7ff28000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:13.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.922+0000 7ff26ffff640 1 -- 172.21.15.163:0/2166686373 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28000f040 con 0x7ff28815c7d0 2024-05-08T06:37:13.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.922+0000 7ff26ffff640 1 -- 172.21.15.163:0/2166686373 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff280004030 con 0x7ff28815c7d0 2024-05-08T06:37:13.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.922+0000 7ff26ffff640 1 -- 172.21.15.163:0/2166686373 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2800043e0 con 0x7ff28815c7d0 2024-05-08T06:37:13.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.922+0000 7ff28dd5f640 1 -- 172.21.15.163:0/2166686373 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2881f59a0 con 0x7ff28815c7d0 2024-05-08T06:37:13.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.922+0000 7ff28dd5f640 1 -- 172.21.15.163:0/2166686373 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2880cf880 con 0x7ff28815c7d0 2024-05-08T06:37:13.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.923+0000 7ff28dd5f640 1 -- 172.21.15.163:0/2166686373 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28806a550 con 0x7ff28815c7d0 2024-05-08T06:37:13.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.923+0000 7ff26ffff640 1 -- 172.21.15.163:0/2166686373 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff28000c6d0 con 0x7ff28815c7d0 2024-05-08T06:37:13.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.924+0000 7ff26ffff640 1 --2- 172.21.15.163:0/2166686373 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff2740762d0 0x7ff274078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:13.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.924+0000 7ff2877fe640 1 --2- 172.21.15.163:0/2166686373 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff2740762d0 0x7ff274078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:13.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.924+0000 7ff26ffff640 1 -- 172.21.15.163:0/2166686373 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7ff280097210 con 0x7ff28815c7d0 2024-05-08T06:37:13.926 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.924+0000 7ff2877fe640 1 --2- 172.21.15.163:0/2166686373 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff2740762d0 0x7ff274078790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff278037000 tx=0x7ff278032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:13.927 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:13.928+0000 7ff26ffff640 1 -- 172.21.15.163:0/2166686373 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff280017020 con 0x7ff28815c7d0 2024-05-08T06:37:14.111 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.112+0000 7ff28dd5f640 1 -- 172.21.15.163:0/2166686373 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff28806f980 con 0x7ff28815c7d0 2024-05-08T06:37:14.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.802+0000 7ff26ffff640 1 -- 172.21.15.163:0/2166686373 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) ==== 181+0+0 (secure 0 0 0) 0x7ff2800604c0 con 0x7ff28815c7d0 2024-05-08T06:37:14.801 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' removed 2024-05-08T06:37:14.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.805+0000 7ff28dd5f640 1 -- 172.21.15.163:0/2166686373 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff2740762d0 msgr2=0x7ff274078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:14.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.805+0000 7ff28dd5f640 1 --2- 172.21.15.163:0/2166686373 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff2740762d0 0x7ff274078790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff278037000 tx=0x7ff278032000 comp rx=0 tx=0).stop 2024-05-08T06:37:14.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.805+0000 7ff28dd5f640 1 -- 172.21.15.163:0/2166686373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28815c7d0 msgr2=0x7ff288154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:14.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.805+0000 7ff28dd5f640 1 --2- 172.21.15.163:0/2166686373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28815c7d0 0x7ff288154e20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff28000e7e0 tx=0x7ff28000eca0 comp rx=0 tx=0).stop 2024-05-08T06:37:14.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.805+0000 7ff28dd5f640 1 -- 172.21.15.163:0/2166686373 shutdown_connections 2024-05-08T06:37:14.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.805+0000 7ff28dd5f640 1 --2- 172.21.15.163:0/2166686373 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff2740762d0 0x7ff274078790 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:14.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.805+0000 7ff28dd5f640 1 --2- 172.21.15.163:0/2166686373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28815c7d0 0x7ff288154e20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:14.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.805+0000 7ff28dd5f640 1 --2- 172.21.15.163:0/2166686373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2880b49f0 0x7ff2881f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:14.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.805+0000 7ff28dd5f640 1 --2- 172.21.15.163:0/2166686373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff28806f010 0x7ff28814ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:14.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.805+0000 7ff28dd5f640 1 -- 172.21.15.163:0/2166686373 >> 172.21.15.163:0/2166686373 conn(0x7ff2880b2b40 msgr2=0x7ff2880723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:14.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.805+0000 7ff28dd5f640 1 -- 172.21.15.163:0/2166686373 shutdown_connections 2024-05-08T06:37:14.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.805+0000 7ff28dd5f640 1 -- 172.21.15.163:0/2166686373 wait complete. 2024-05-08T06:37:14.819 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T06:37:14.928 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.929+0000 7fd1333dd640 1 Processor -- start 2024-05-08T06:37:14.928 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.929+0000 7fd1333dd640 1 -- start start 2024-05-08T06:37:14.930 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.930+0000 7fd1333dd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd12c070e50 0x7fd12c071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:14.930 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.930+0000 7fd1333dd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd12c0cf5a0 0x7fd12c06e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:14.930 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.930+0000 7fd1333dd640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd12c0b7b20 con 0x7fd12c0d3b60 2024-05-08T06:37:14.930 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.930+0000 7fd1333dd640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fd12c0b82b0 con 0x7fd12c070e50 2024-05-08T06:37:14.930 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.930+0000 7fd1333dd640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd12c070900 con 0x7fd12c0cf5a0 2024-05-08T06:37:14.930 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.930+0000 7fd131179640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd12c070e50 0x7fd12c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:14.930 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.930+0000 7fd131179640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd12c070e50 0x7fd12c071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40638/0 (socket says 172.21.15.163:40638) 2024-05-08T06:37:14.930 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.930+0000 7fd131179640 1 -- 172.21.15.163:0/4240669623 learned_addr learned my addr 172.21.15.163:0/4240669623 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:14.930 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.931+0000 7fd131179640 1 -- 172.21.15.163:0/4240669623 >> v2:172.21.15.192:3300/0 conn(0x7fd12c0cf5a0 msgr2=0x7fd12c06e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:14.931 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.931+0000 7fd13197a640 1 --2- 172.21.15.163:0/4240669623 >> v2:172.21.15.192:3300/0 conn(0x7fd12c0cf5a0 0x7fd12c06e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:14.931 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.931+0000 7fd131179640 1 --2- 172.21.15.163:0/4240669623 >> v2:172.21.15.192:3300/0 conn(0x7fd12c0cf5a0 0x7fd12c06e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:14.931 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.931+0000 7fd131179640 1 -- 172.21.15.163:0/4240669623 >> v1:172.21.15.192:6789/0 conn(0x7fd12c0d3b60 legacy=0x7fd12c06de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:14.931 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.931+0000 7fd131179640 1 -- 172.21.15.163:0/4240669623 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12c0aff30 con 0x7fd12c070e50 2024-05-08T06:37:14.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.931+0000 7fd131179640 1 --2- 172.21.15.163:0/4240669623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12c070e50 0x7fd12c071230 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd12c06d2b0 tx=0x7fd12802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1705ae410fcffb0e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:14.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.932+0000 7fd123fff640 1 -- 172.21.15.163:0/4240669623 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128037070 con 0x7fd12c070e50 2024-05-08T06:37:14.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.932+0000 7fd123fff640 1 -- 172.21.15.163:0/4240669623 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd128032440 con 0x7fd12c070e50 2024-05-08T06:37:14.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.932+0000 7fd123fff640 1 -- 172.21.15.163:0/4240669623 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128032810 con 0x7fd12c070e50 2024-05-08T06:37:14.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.932+0000 7fd1333dd640 1 -- 172.21.15.163:0/4240669623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12c070e50 msgr2=0x7fd12c071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:14.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.932+0000 7fd1333dd640 1 --2- 172.21.15.163:0/4240669623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12c070e50 0x7fd12c071230 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd12c06d2b0 tx=0x7fd12802a1e0 comp rx=0 tx=0).stop 2024-05-08T06:37:14.933 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.932+0000 7fd1333dd640 1 -- 172.21.15.163:0/4240669623 shutdown_connections 2024-05-08T06:37:14.933 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.932+0000 7fd1333dd640 1 --2- 172.21.15.163:0/4240669623 >> v2:172.21.15.192:3300/0 conn(0x7fd12c0cf5a0 0x7fd12c06e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:14.933 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.932+0000 7fd1333dd640 1 --2- 172.21.15.163:0/4240669623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12c070e50 0x7fd12c071230 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:14.933 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.932+0000 7fd1333dd640 1 -- 172.21.15.163:0/4240669623 >> 172.21.15.163:0/4240669623 conn(0x7fd12c0d3040 msgr2=0x7fd12c0d3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:14.933 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.932+0000 7fd1333dd640 1 -- 172.21.15.163:0/4240669623 shutdown_connections 2024-05-08T06:37:14.933 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.932+0000 7fd1333dd640 1 -- 172.21.15.163:0/4240669623 wait complete. 2024-05-08T06:37:14.933 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.933+0000 7fd1333dd640 1 Processor -- start 2024-05-08T06:37:14.933 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.933+0000 7fd1333dd640 1 -- start start 2024-05-08T06:37:14.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.934+0000 7fd1333dd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd12c070e50 0x7fd12c09d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:14.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.934+0000 7fd1333dd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12c0cf5a0 0x7fd12c15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:14.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.934+0000 7fd1333dd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd12c0d3b60 0x7fd12c09ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:14.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.934+0000 7fd1333dd640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c071a20 con 0x7fd12c0cf5a0 2024-05-08T06:37:14.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.934+0000 7fd1333dd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c0b7e20 con 0x7fd12c070e50 2024-05-08T06:37:14.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.934+0000 7fd1333dd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd12c071d00 con 0x7fd12c0d3b60 2024-05-08T06:37:14.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.934+0000 7fd130978640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12c0cf5a0 0x7fd12c15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:14.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.934+0000 7fd130978640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12c0cf5a0 0x7fd12c15ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40648/0 (socket says 172.21.15.163:40648) 2024-05-08T06:37:14.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.934+0000 7fd130978640 1 -- 172.21.15.163:0/1069332074 learned_addr learned my addr 172.21.15.163:0/1069332074 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:14.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.934+0000 7fd131179640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd12c070e50 0x7fd12c09d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:14.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.934+0000 7fd13197a640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd12c0d3b60 0x7fd12c09ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:14.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.935+0000 7fd130978640 1 -- 172.21.15.163:0/1069332074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd12c0d3b60 msgr2=0x7fd12c09ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:14.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.935+0000 7fd130978640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd12c0d3b60 0x7fd12c09ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:14.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.935+0000 7fd130978640 1 -- 172.21.15.163:0/1069332074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd12c070e50 msgr2=0x7fd12c09d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:14.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.935+0000 7fd130978640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd12c070e50 0x7fd12c09d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:14.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.935+0000 7fd130978640 1 -- 172.21.15.163:0/1069332074 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12c1590e0 con 0x7fd12c0cf5a0 2024-05-08T06:37:14.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.935+0000 7fd13197a640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd12c0d3b60 0x7fd12c09ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:14.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.935+0000 7fd130978640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12c0cf5a0 0x7fd12c15ba90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00c440 tx=0x7fd11c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:14.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.935+0000 7fd121ffb640 1 -- 172.21.15.163:0/1069332074 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c017070 con 0x7fd12c0cf5a0 2024-05-08T06:37:14.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.935+0000 7fd121ffb640 1 -- 172.21.15.163:0/1069332074 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd11c0035f0 con 0x7fd12c0cf5a0 2024-05-08T06:37:14.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.935+0000 7fd1333dd640 1 -- 172.21.15.163:0/1069332074 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12c1593c0 con 0x7fd12c0cf5a0 2024-05-08T06:37:14.937 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.935+0000 7fd121ffb640 1 -- 172.21.15.163:0/1069332074 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0039e0 con 0x7fd12c0cf5a0 2024-05-08T06:37:14.937 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.935+0000 7fd1333dd640 1 -- 172.21.15.163:0/1069332074 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12c159980 con 0x7fd12c0cf5a0 2024-05-08T06:37:14.937 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.936+0000 7fd1333dd640 1 -- 172.21.15.163:0/1069332074 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd12c06a550 con 0x7fd12c0cf5a0 2024-05-08T06:37:14.937 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.937+0000 7fd121ffb640 1 -- 172.21.15.163:0/1069332074 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd11c01a070 con 0x7fd12c0cf5a0 2024-05-08T06:37:14.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.937+0000 7fd121ffb640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd1100762d0 0x7fd110078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:14.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.937+0000 7fd121ffb640 1 -- 172.21.15.163:0/1069332074 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5357+0+0 (secure 0 0 0) 0x7fd11c0967c0 con 0x7fd12c0cf5a0 2024-05-08T06:37:14.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.937+0000 7fd131179640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd1100762d0 0x7fd110078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:14.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.938+0000 7fd131179640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd1100762d0 0x7fd110078790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd128034000 tx=0x7fd12802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:14.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:14.941+0000 7fd121ffb640 1 -- 172.21.15.163:0/1069332074 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd11c060310 con 0x7fd12c0cf5a0 2024-05-08T06:37:15.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.126+0000 7fd1333dd640 1 -- 172.21.15.163:0/1069332074 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd12c06c7b0 con 0x7fd12c0cf5a0 2024-05-08T06:37:15.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.807+0000 7fd121ffb640 1 -- 172.21.15.163:0/1069332074 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) ==== 169+0+0 (secure 0 0 0) 0x7fd11c0010c0 con 0x7fd12c0cf5a0 2024-05-08T06:37:15.806 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' removed 2024-05-08T06:37:15.810 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.811+0000 7fd1333dd640 1 -- 172.21.15.163:0/1069332074 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd1100762d0 msgr2=0x7fd110078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:15.810 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.811+0000 7fd1333dd640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd1100762d0 0x7fd110078790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd128034000 tx=0x7fd12802e000 comp rx=0 tx=0).stop 2024-05-08T06:37:15.811 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.811+0000 7fd1333dd640 1 -- 172.21.15.163:0/1069332074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12c0cf5a0 msgr2=0x7fd12c15ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:15.811 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.811+0000 7fd1333dd640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12c0cf5a0 0x7fd12c15ba90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00c440 tx=0x7fd11c00c900 comp rx=0 tx=0).stop 2024-05-08T06:37:15.811 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.811+0000 7fd1333dd640 1 -- 172.21.15.163:0/1069332074 shutdown_connections 2024-05-08T06:37:15.811 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.811+0000 7fd1333dd640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd1100762d0 0x7fd110078790 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:15.811 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.811+0000 7fd1333dd640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd12c0d3b60 0x7fd12c09ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:15.811 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.811+0000 7fd1333dd640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd12c0cf5a0 0x7fd12c15ba90 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00c440 tx=0x7fd11c00c900 comp rx=0 tx=0).stop 2024-05-08T06:37:15.811 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.811+0000 7fd1333dd640 1 --2- 172.21.15.163:0/1069332074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd12c070e50 0x7fd12c09d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:15.811 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.811+0000 7fd1333dd640 1 -- 172.21.15.163:0/1069332074 >> 172.21.15.163:0/1069332074 conn(0x7fd12c0d3040 msgr2=0x7fd12c06de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:15.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.812+0000 7fd1333dd640 1 -- 172.21.15.163:0/1069332074 shutdown_connections 2024-05-08T06:37:15.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.812+0000 7fd1333dd640 1 -- 172.21.15.163:0/1069332074 wait complete. 2024-05-08T06:37:15.825 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T06:37:15.825 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T06:37:15.825 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T06:37:15.825 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:37:15.826 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T06:37:15.826 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T06:37:15.826 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T06:37:15.826 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:37:15.826 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T06:37:15.826 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T06:37:15.827 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T06:37:15.827 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:37:15.827 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T06:37:15.827 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T06:37:15.827 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T06:37:15.827 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:37:15.836 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T06:37:15.836 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T06:37:15.837 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T06:37:15.837 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T06:37:15.839 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T06:37:15.839 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T06:37:15.839 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T06:37:15.840 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T06:37:15.879 INFO:tasks.ceph.mds.c:Started 2024-05-08T06:37:15.880 DEBUG:teuthology.parallel:result is None 2024-05-08T06:37:15.880 INFO:tasks.ceph.mds.a:Started 2024-05-08T06:37:15.880 DEBUG:teuthology.parallel:result is None 2024-05-08T06:37:15.883 INFO:tasks.ceph.mds.b:Started 2024-05-08T06:37:15.883 DEBUG:teuthology.parallel:result is None 2024-05-08T06:37:15.884 INFO:tasks.ceph.mds.d:Started 2024-05-08T06:37:15.884 DEBUG:teuthology.parallel:result is None 2024-05-08T06:37:15.884 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T06:37:15.959 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2024-05-08T06:37:15.959 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2024-05-08T06:37:15.968 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-08T06:37:15.969 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-08T06:37:15.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.991+0000 7fa9cac37640 1 Processor -- start 2024-05-08T06:37:15.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.991+0000 7fa9cac37640 1 -- start start 2024-05-08T06:37:15.993 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9cac37640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa9c415c150 0x7fa9c40b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:15.993 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9cac37640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c40b12f0 0x7fa9c40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:15.993 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9cac37640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa9c40b5470 0x7fa9c4073480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:15.993 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9cac37640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa9c40afd50 con 0x7fa9c415c150 2024-05-08T06:37:15.993 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9cac37640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa9c415c530 con 0x7fa9c40b5470 2024-05-08T06:37:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9cac37640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c40afff0 con 0x7fa9c40b12f0 2024-05-08T06:37:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9c89d3640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa9c415c150 0x7fa9c40b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9c89d3640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa9c415c150 0x7fa9c40b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40672/0 (socket says 172.21.15.163:40672) 2024-05-08T06:37:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9c89d3640 1 -- 172.21.15.163:0/1589439860 learned_addr learned my addr 172.21.15.163:0/1589439860 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9c91d4640 1 --2- 172.21.15.163:0/1589439860 >> v2:172.21.15.192:3300/0 conn(0x7fa9c40b5470 0x7fa9c4073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9c3fff640 1 --2- 172.21.15.163:0/1589439860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c40b12f0 0x7fa9c40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9c89d3640 1 -- 172.21.15.163:0/1589439860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c40b12f0 msgr2=0x7fa9c40b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9c89d3640 1 --2- 172.21.15.163:0/1589439860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c40b12f0 0x7fa9c40b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9c89d3640 1 -- 172.21.15.163:0/1589439860 >> v2:172.21.15.192:3300/0 conn(0x7fa9c40b5470 msgr2=0x7fa9c4073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9c89d3640 1 --2- 172.21.15.163:0/1589439860 >> v2:172.21.15.192:3300/0 conn(0x7fa9c40b5470 0x7fa9c4073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9c89d3640 1 -- 172.21.15.163:0/1589439860 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c40b02d0 con 0x7fa9c415c150 2024-05-08T06:37:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.992+0000 7fa9c3fff640 1 --2- 172.21.15.163:0/1589439860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c40b12f0 0x7fa9c40b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.993+0000 7fa9c89d3640 1 --2- 172.21.15.163:0/1589439860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9c415c150 0x7fa9c40b0db0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40d3010 tx=0x7fa9b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4e2cc15b82c0ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.993+0000 7fa9c37fe640 1 -- 172.21.15.163:0/1589439860 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8037070 con 0x7fa9c415c150 2024-05-08T06:37:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.993+0000 7fa9c37fe640 1 -- 172.21.15.163:0/1589439860 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9b8032440 con 0x7fa9c415c150 2024-05-08T06:37:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.993+0000 7fa9c37fe640 1 -- 172.21.15.163:0/1589439860 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8032810 con 0x7fa9c415c150 2024-05-08T06:37:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.994+0000 7fa9cac37640 1 -- 172.21.15.163:0/1589439860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9c415c150 msgr2=0x7fa9c40b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.994+0000 7fa9cac37640 1 --2- 172.21.15.163:0/1589439860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9c415c150 0x7fa9c40b0db0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40d3010 tx=0x7fa9b802a1e0 comp rx=0 tx=0).stop 2024-05-08T06:37:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.994+0000 7fa9cac37640 1 -- 172.21.15.163:0/1589439860 shutdown_connections 2024-05-08T06:37:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.994+0000 7fa9cac37640 1 --2- 172.21.15.163:0/1589439860 >> v2:172.21.15.192:3300/0 conn(0x7fa9c40b5470 0x7fa9c4073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.994+0000 7fa9cac37640 1 --2- 172.21.15.163:0/1589439860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c40b12f0 0x7fa9c40b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.994+0000 7fa9cac37640 1 --2- 172.21.15.163:0/1589439860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9c415c150 0x7fa9c40b0db0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.994+0000 7fa9cac37640 1 -- 172.21.15.163:0/1589439860 >> 172.21.15.163:0/1589439860 conn(0x7fa9c40b3c30 msgr2=0x7fa9c40b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.994+0000 7fa9cac37640 1 -- 172.21.15.163:0/1589439860 shutdown_connections 2024-05-08T06:37:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.994+0000 7fa9cac37640 1 -- 172.21.15.163:0/1589439860 wait complete. 2024-05-08T06:37:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.995+0000 7fa9cac37640 1 Processor -- start 2024-05-08T06:37:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.995+0000 7fa9cac37640 1 -- start start 2024-05-08T06:37:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9cac37640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9c40b12f0 0x7fa9c414cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9cac37640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9c40b5470 0x7fa9c415ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9cac37640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c415c150 0x7fa9c406f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9cac37640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c40739c0 con 0x7fa9c40b12f0 2024-05-08T06:37:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9cac37640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c40b0890 con 0x7fa9c40b5470 2024-05-08T06:37:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9cac37640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c4073b40 con 0x7fa9c415c150 2024-05-08T06:37:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9c89d3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9c40b12f0 0x7fa9c414cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9c89d3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9c40b12f0 0x7fa9c414cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40686/0 (socket says 172.21.15.163:40686) 2024-05-08T06:37:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9c89d3640 1 -- 172.21.15.163:0/2128793189 learned_addr learned my addr 172.21.15.163:0/2128793189 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9c91d4640 1 --2- 172.21.15.163:0/2128793189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c415c150 0x7fa9c406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9c3fff640 1 --2- 172.21.15.163:0/2128793189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9c40b5470 0x7fa9c415ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9c89d3640 1 -- 172.21.15.163:0/2128793189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c415c150 msgr2=0x7fa9c406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9c89d3640 1 --2- 172.21.15.163:0/2128793189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c415c150 0x7fa9c406f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9c89d3640 1 -- 172.21.15.163:0/2128793189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9c40b5470 msgr2=0x7fa9c415ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9c89d3640 1 --2- 172.21.15.163:0/2128793189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9c40b5470 0x7fa9c415ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.996+0000 7fa9c89d3640 1 -- 172.21.15.163:0/2128793189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c415f3f0 con 0x7fa9c40b12f0 2024-05-08T06:37:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.997+0000 7fa9c89d3640 1 --2- 172.21.15.163:0/2128793189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9c40b12f0 0x7fa9c414cce0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa9b802a220 tx=0x7fa9b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.997+0000 7fa9c17fa640 1 -- 172.21.15.163:0/2128793189 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8041020 con 0x7fa9c40b12f0 2024-05-08T06:37:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.997+0000 7fa9c17fa640 1 -- 172.21.15.163:0/2128793189 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9b80306c0 con 0x7fa9c40b12f0 2024-05-08T06:37:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.997+0000 7fa9c17fa640 1 -- 172.21.15.163:0/2128793189 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8037040 con 0x7fa9c40b12f0 2024-05-08T06:37:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.997+0000 7fa9cac37640 1 -- 172.21.15.163:0/2128793189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c409d610 con 0x7fa9c40b12f0 2024-05-08T06:37:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.997+0000 7fa9cac37640 1 -- 172.21.15.163:0/2128793189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c409dbd0 con 0x7fa9c40b12f0 2024-05-08T06:37:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.998+0000 7fa9cac37640 1 -- 172.21.15.163:0/2128793189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c406a4b0 con 0x7fa9c40b12f0 2024-05-08T06:37:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.998+0000 7fa9c17fa640 1 -- 172.21.15.163:0/2128793189 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa9b8044020 con 0x7fa9c40b12f0 2024-05-08T06:37:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.999+0000 7fa9c17fa640 1 --2- 172.21.15.163:0/2128793189 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa9a80762d0 0x7fa9a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.999+0000 7fa9c3fff640 1 --2- 172.21.15.163:0/2128793189 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa9a80762d0 0x7fa9a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:15.999+0000 7fa9c17fa640 1 -- 172.21.15.163:0/2128793189 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4943+0+0 (secure 0 0 0) 0x7fa9b80b7950 con 0x7fa9c40b12f0 2024-05-08T06:37:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.000+0000 7fa9c3fff640 1 --2- 172.21.15.163:0/2128793189 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa9a80762d0 0x7fa9a8078790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa9c415b050 tx=0x7fa9b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:16.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.003+0000 7fa9c17fa640 1 -- 172.21.15.163:0/2128793189 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa9b804b030 con 0x7fa9c40b12f0 2024-05-08T06:37:16.202 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.203+0000 7fa9cac37640 1 -- 172.21.15.163:0/2128793189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa9c40aef10 con 0x7fa9c40b12f0 2024-05-08T06:37:16.815 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.816+0000 7fa9c17fa640 1 -- 172.21.15.163:0/2128793189 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) ==== 97+0+0 (secure 0 0 0) 0x7fa9b8080ff0 con 0x7fa9c40b12f0 2024-05-08T06:37:16.816 INFO:teuthology.orchestra.run.smithi163.stderr: removed all blocklist entries 2024-05-08T06:37:16.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.819+0000 7fa9cac37640 1 -- 172.21.15.163:0/2128793189 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa9a80762d0 msgr2=0x7fa9a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:16.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.819+0000 7fa9cac37640 1 --2- 172.21.15.163:0/2128793189 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa9a80762d0 0x7fa9a8078790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa9c415b050 tx=0x7fa9b0008040 comp rx=0 tx=0).stop 2024-05-08T06:37:16.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.819+0000 7fa9cac37640 1 -- 172.21.15.163:0/2128793189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9c40b12f0 msgr2=0x7fa9c414cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:16.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.820+0000 7fa9cac37640 1 --2- 172.21.15.163:0/2128793189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9c40b12f0 0x7fa9c414cce0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa9b802a220 tx=0x7fa9b80046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:16.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.820+0000 7fa9cac37640 1 -- 172.21.15.163:0/2128793189 shutdown_connections 2024-05-08T06:37:16.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.820+0000 7fa9cac37640 1 --2- 172.21.15.163:0/2128793189 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa9a80762d0 0x7fa9a8078790 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:16.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.820+0000 7fa9cac37640 1 --2- 172.21.15.163:0/2128793189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c415c150 0x7fa9c406f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:16.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.820+0000 7fa9cac37640 1 --2- 172.21.15.163:0/2128793189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9c40b5470 0x7fa9c415ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:16.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.820+0000 7fa9cac37640 1 --2- 172.21.15.163:0/2128793189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9c40b12f0 0x7fa9c414cce0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:16.820 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.820+0000 7fa9cac37640 1 -- 172.21.15.163:0/2128793189 >> 172.21.15.163:0/2128793189 conn(0x7fa9c40b3c30 msgr2=0x7fa9c40b21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:16.820 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.820+0000 7fa9cac37640 1 -- 172.21.15.163:0/2128793189 shutdown_connections 2024-05-08T06:37:16.820 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.820+0000 7fa9cac37640 1 -- 172.21.15.163:0/2128793189 wait complete. 2024-05-08T06:37:16.832 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T06:37:16.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.935+0000 7f2b2f264640 1 Processor -- start 2024-05-08T06:37:16.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.935+0000 7f2b2f264640 1 -- start start 2024-05-08T06:37:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.936+0000 7f2b2f264640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b280d3640 0x7f2b280d4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.936+0000 7f2b2f264640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b280d0040 0x7f2b280d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.936+0000 7f2b2f264640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b280d3b20 con 0x7f2b2815f410 2024-05-08T06:37:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.936+0000 7f2b2f264640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f2b280d3e00 con 0x7f2b280d3640 2024-05-08T06:37:16.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.936+0000 7f2b2f264640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2b280d40e0 con 0x7f2b280d0040 2024-05-08T06:37:16.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.936+0000 7f2b1ffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b280d3640 0x7f2b280d4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:16.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.936+0000 7f2b2d000640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2b2815f410 0x7f2b280bab90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60602/0 (socket says 172.21.15.163:60602) 2024-05-08T06:37:16.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.936+0000 7f2b2d000640 1 -- 172.21.15.163:0/1772588572 learned_addr learned my addr 172.21.15.163:0/1772588572 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:16.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.936+0000 7f2b2d801640 1 --2- 172.21.15.163:0/1772588572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b280d0040 0x7f2b280d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:16.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.936+0000 7f2b1ffff640 1 -- 172.21.15.163:0/1772588572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b280d0040 msgr2=0x7f2b280d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:16.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.936+0000 7f2b1ffff640 1 --2- 172.21.15.163:0/1772588572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b280d0040 0x7f2b280d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:16.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.936+0000 7f2b1ffff640 1 -- 172.21.15.163:0/1772588572 >> v1:172.21.15.163:6789/0 conn(0x7f2b2815f410 legacy=0x7f2b280bab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:16.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.936+0000 7f2b1f7fe640 1 -- 172.21.15.163:0/1772588572 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 343358214 0 0) 0x7f2b280d3b20 con 0x7f2b2815f410 2024-05-08T06:37:16.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.936+0000 7f2b1ffff640 1 -- 172.21.15.163:0/1772588572 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b280d43c0 con 0x7f2b280d3640 2024-05-08T06:37:16.937 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.937+0000 7f2b1ffff640 1 --2- 172.21.15.163:0/1772588572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280d3640 0x7f2b280d4ba0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2b180097a0 tx=0x7f2b1802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a4c831c814e55d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:16.937 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.937+0000 7f2b1f7fe640 1 -- 172.21.15.163:0/1772588572 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18037070 con 0x7f2b280d3640 2024-05-08T06:37:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.937+0000 7f2b1f7fe640 1 -- 172.21.15.163:0/1772588572 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b18032440 con 0x7f2b280d3640 2024-05-08T06:37:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.937+0000 7f2b1f7fe640 1 -- 172.21.15.163:0/1772588572 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18032810 con 0x7f2b280d3640 2024-05-08T06:37:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.938+0000 7f2b2f264640 1 -- 172.21.15.163:0/1772588572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280d3640 msgr2=0x7f2b280d4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.938+0000 7f2b2f264640 1 --2- 172.21.15.163:0/1772588572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280d3640 0x7f2b280d4ba0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2b180097a0 tx=0x7f2b1802a180 comp rx=0 tx=0).stop 2024-05-08T06:37:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.938+0000 7f2b2f264640 1 -- 172.21.15.163:0/1772588572 shutdown_connections 2024-05-08T06:37:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.938+0000 7f2b2f264640 1 --2- 172.21.15.163:0/1772588572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b280d0040 0x7f2b280d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.938+0000 7f2b2f264640 1 --2- 172.21.15.163:0/1772588572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280d3640 0x7f2b280d4ba0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.938+0000 7f2b2f264640 1 -- 172.21.15.163:0/1772588572 >> 172.21.15.163:0/1772588572 conn(0x7f2b28073e40 msgr2=0x7f2b28074250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.938+0000 7f2b2f264640 1 -- 172.21.15.163:0/1772588572 shutdown_connections 2024-05-08T06:37:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.938+0000 7f2b2f264640 1 -- 172.21.15.163:0/1772588572 wait complete. 2024-05-08T06:37:16.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.939+0000 7f2b2f264640 1 Processor -- start 2024-05-08T06:37:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.939+0000 7f2b2f264640 1 -- start start 2024-05-08T06:37:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b2f264640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b280d0040 0x7f2b2814e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b2f264640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280d3640 0x7f2b2814d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b2f264640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b2815f410 0x7f2b280abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b2f264640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2815f970 con 0x7f2b280d3640 2024-05-08T06:37:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b2f264640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2815f7f0 con 0x7f2b280d0040 2024-05-08T06:37:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b2f264640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2b280b83a0 con 0x7f2b2815f410 2024-05-08T06:37:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b1ffff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280d3640 0x7f2b2814d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b1ffff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280d3640 0x7f2b2814d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40690/0 (socket says 172.21.15.163:40690) 2024-05-08T06:37:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b1ffff640 1 -- 172.21.15.163:0/3862201859 learned_addr learned my addr 172.21.15.163:0/3862201859 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b2d000640 1 --2- 172.21.15.163:0/3862201859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b280d0040 0x7f2b2814e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b2d801640 1 --2- 172.21.15.163:0/3862201859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b2815f410 0x7f2b280abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:16.942 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b1ffff640 1 -- 172.21.15.163:0/3862201859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b2815f410 msgr2=0x7f2b280abeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:16.942 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b1ffff640 1 --2- 172.21.15.163:0/3862201859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b2815f410 0x7f2b280abeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:16.942 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b1ffff640 1 -- 172.21.15.163:0/3862201859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b280d0040 msgr2=0x7f2b2814e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:16.942 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b1ffff640 1 --2- 172.21.15.163:0/3862201859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b280d0040 0x7f2b2814e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:16.942 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.940+0000 7f2b1ffff640 1 -- 172.21.15.163:0/3862201859 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2814e870 con 0x7f2b280d3640 2024-05-08T06:37:16.942 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.941+0000 7f2b1ffff640 1 --2- 172.21.15.163:0/3862201859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280d3640 0x7f2b2814d9b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2b18035040 tx=0x7f2b180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:16.942 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.941+0000 7f2b1d7fa640 1 -- 172.21.15.163:0/3862201859 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18037050 con 0x7f2b280d3640 2024-05-08T06:37:16.942 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.941+0000 7f2b1d7fa640 1 -- 172.21.15.163:0/3862201859 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b18038040 con 0x7f2b280d3640 2024-05-08T06:37:16.942 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.941+0000 7f2b1d7fa640 1 -- 172.21.15.163:0/3862201859 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18004490 con 0x7f2b280d3640 2024-05-08T06:37:16.942 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.941+0000 7f2b2f264640 1 -- 172.21.15.163:0/3862201859 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2814eb50 con 0x7f2b280d3640 2024-05-08T06:37:16.942 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.941+0000 7f2b2f264640 1 -- 172.21.15.163:0/3862201859 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b2814f110 con 0x7f2b280d3640 2024-05-08T06:37:16.942 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.942+0000 7f2b2f264640 1 -- 172.21.15.163:0/3862201859 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b280d3b20 con 0x7f2b280d3640 2024-05-08T06:37:16.943 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.942+0000 7f2b1d7fa640 1 -- 172.21.15.163:0/3862201859 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2b18030900 con 0x7f2b280d3640 2024-05-08T06:37:16.943 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.943+0000 7f2b1d7fa640 1 --2- 172.21.15.163:0/3862201859 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2b140762d0 0x7f2b14078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:16.943 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.943+0000 7f2b1d7fa640 1 -- 172.21.15.163:0/3862201859 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f2b180b7cd0 con 0x7f2b280d3640 2024-05-08T06:37:16.943 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.943+0000 7f2b2d000640 1 --2- 172.21.15.163:0/3862201859 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2b140762d0 0x7f2b14078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:16.943 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.943+0000 7f2b2d000640 1 --2- 172.21.15.163:0/3862201859 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2b140762d0 0x7f2b14078790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2b240012e0 tx=0x7f2b24001630 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:16.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:16.947+0000 7f2b1d7fa640 1 -- 172.21.15.163:0/3862201859 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2b280d3b20 con 0x7f2b280d3640 2024-05-08T06:37:17.200 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.201+0000 7f2b2f264640 1 -- 172.21.15.163:0/3862201859 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f2b2815e480 con 0x7f2b280d3640 2024-05-08T06:37:17.201 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.202+0000 7f2b1d7fa640 1 -- 172.21.15.163:0/3862201859 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7f2b2815e480 con 0x7f2b280d3640 2024-05-08T06:37:17.201 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:17.201 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:37:17.201 INFO:teuthology.orchestra.run.smithi163.stdout: "auth_dump": [ 2024-05-08T06:37:17.202 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.202 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.a", 2024-05-08T06:37:17.202 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBAHTtmqmfZDBAANo1hg5zAsXYQCFggPgdDrg==", 2024-05-08T06:37:17.202 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.202 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2024-05-08T06:37:17.202 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2024-05-08T06:37:17.202 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2024-05-08T06:37:17.202 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.202 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.202 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.202 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.202 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.b", 2024-05-08T06:37:17.202 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBAHTtmUgCVGBAApW+5C4VEymZvs5rTTSRpag==", 2024-05-08T06:37:17.203 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.203 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2024-05-08T06:37:17.203 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2024-05-08T06:37:17.203 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2024-05-08T06:37:17.203 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.203 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.203 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.203 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.203 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.c", 2024-05-08T06:37:17.203 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBAHTtmxMbxExAAGlrRx6CvlReC6EMej743Ug==", 2024-05-08T06:37:17.203 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.203 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2024-05-08T06:37:17.203 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2024-05-08T06:37:17.203 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.d", 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBAHTtmPqmZHxAAmJ4mxC6u5Gq1STQhAFymPQ==", 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.204 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.0", 2024-05-08T06:37:17.205 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBEHTtmqdDbAxAALZBFHAbE7N680+1LMNdFpQ==", 2024-05-08T06:37:17.205 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.205 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2024-05-08T06:37:17.205 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2024-05-08T06:37:17.205 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.205 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.205 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.205 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.206 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.1", 2024-05-08T06:37:17.206 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBHHTtmvs0CAhAASCqBWYMADdQ18jnz4Em0mg==", 2024-05-08T06:37:17.206 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.206 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2024-05-08T06:37:17.206 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2024-05-08T06:37:17.206 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.206 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.206 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.206 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.206 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.2", 2024-05-08T06:37:17.206 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBKHTtmj/QUABAAZo33r3p9GzUGIxrtfCx1Gg==", 2024-05-08T06:37:17.206 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.206 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2024-05-08T06:37:17.207 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2024-05-08T06:37:17.207 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.207 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.207 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.207 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.207 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.3", 2024-05-08T06:37:17.207 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBMHTtmT/6EOhAA+DZAI4ARReOZuSOa8prIdA==", 2024-05-08T06:37:17.207 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.207 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2024-05-08T06:37:17.207 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2024-05-08T06:37:17.207 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.207 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.207 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.207 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.4", 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBTHTtmVkRmAxAAP/TBnP/iNQzmTk7UOZEX5w==", 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.5", 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBWHTtm8AygARAAtRm5MEdIkMZAokmbzJHqlQ==", 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2024-05-08T06:37:17.208 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.209 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.209 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.209 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.209 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.6", 2024-05-08T06:37:17.209 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBYHTtmuVqLOxAA7QYDS8WD1C6MJk/LxgvS1A==", 2024-05-08T06:37:17.209 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.209 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2024-05-08T06:37:17.209 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2024-05-08T06:37:17.209 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.209 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.209 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.209 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.209 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.7", 2024-05-08T06:37:17.209 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBbHTtmHfg8ORAAGcnBJycy8okW7aEJMY3pug==", 2024-05-08T06:37:17.210 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.210 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2024-05-08T06:37:17.210 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2024-05-08T06:37:17.210 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.210 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.210 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.214 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.214 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.0", 2024-05-08T06:37:17.214 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBAHTtmobegIxAApWbSIBzhwZQyiH9lGjNodA==", 2024-05-08T06:37:17.214 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.214 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2024-05-08T06:37:17.214 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2024-05-08T06:37:17.214 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2024-05-08T06:37:17.214 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2024-05-08T06:37:17.214 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.215 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.215 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.215 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.1", 2024-05-08T06:37:17.215 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBAHTtmzf5cLBAAkDg+s+PCS5L6wwtbtpBcbw==", 2024-05-08T06:37:17.215 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.215 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2024-05-08T06:37:17.215 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2024-05-08T06:37:17.215 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2024-05-08T06:37:17.215 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2024-05-08T06:37:17.215 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.215 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.215 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.215 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.2", 2024-05-08T06:37:17.216 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBAHTtmhewvKRAAv6ild817yPYN5MT8UQOlqg==", 2024-05-08T06:37:17.216 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.216 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2024-05-08T06:37:17.216 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2024-05-08T06:37:17.216 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2024-05-08T06:37:17.216 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2024-05-08T06:37:17.216 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.216 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.216 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.216 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.3", 2024-05-08T06:37:17.216 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBAHTtmaITuMRAAVnocUQRC+LPUV6hDfKBblA==", 2024-05-08T06:37:17.216 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.216 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2024-05-08T06:37:17.217 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2024-05-08T06:37:17.217 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2024-05-08T06:37:17.217 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2024-05-08T06:37:17.217 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.217 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.217 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.217 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.admin", 2024-05-08T06:37:17.217 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQA/HTtm/GPPLBAApIJnN3k98kLOADwPkv7okw==", 2024-05-08T06:37:17.217 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.217 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2024-05-08T06:37:17.217 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2024-05-08T06:37:17.217 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2024-05-08T06:37:17.217 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.217 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.218 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.218 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.218 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.bootstrap-mds", 2024-05-08T06:37:17.218 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBhHTtmDu7CLBAAKXCnD/EtCJ4miIefFSEarg==", 2024-05-08T06:37:17.218 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.218 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile bootstrap-mds" 2024-05-08T06:37:17.218 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.218 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.218 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.218 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.bootstrap-mgr", 2024-05-08T06:37:17.218 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBhHTtm2vXCLBAA/crRulWRUyU6AvELmKQRBA==", 2024-05-08T06:37:17.218 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.218 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-08T06:37:17.218 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.bootstrap-osd", 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBhHTtmPv3CLBAAHAjEZwRxsSElrHOx0C21BA==", 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile bootstrap-osd" 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.bootstrap-rbd", 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBhHTtm7wPDLBAA27VpiL/pODxKXkpqI+NloQ==", 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.219 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBhHTtmtQrDLBAAf/IoYl61P5ZYUuZc7SdZWg==", 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.bootstrap-rgw", 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBhHTtmshHDLBAAfyc4B2zdvFoSMIlb96RZsA==", 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.220 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.221 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.x", 2024-05-08T06:37:17.221 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBAHTtmzk6gCRAABCm0Mn6GBZL/M7B+kmzhVA==", 2024-05-08T06:37:17.221 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.221 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2024-05-08T06:37:17.221 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2024-05-08T06:37:17.221 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.221 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.221 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:17.221 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:17.221 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.y", 2024-05-08T06:37:17.221 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQBAHTtmkLEUBhAAsMQc6/tPdPXIPV+iJTyvNg==", 2024-05-08T06:37:17.221 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2024-05-08T06:37:17.221 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2024-05-08T06:37:17.221 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2024-05-08T06:37:17.222 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2024-05-08T06:37:17.222 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.222 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:17.222 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2024-05-08T06:37:17.222 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:37:17.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.204+0000 7f2b2f264640 1 -- 172.21.15.163:0/3862201859 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2b140762d0 msgr2=0x7f2b14078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.204+0000 7f2b2f264640 1 --2- 172.21.15.163:0/3862201859 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2b140762d0 0x7f2b14078790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2b240012e0 tx=0x7f2b24001630 comp rx=0 tx=0).stop 2024-05-08T06:37:17.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.204+0000 7f2b2f264640 1 -- 172.21.15.163:0/3862201859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280d3640 msgr2=0x7f2b2814d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.204+0000 7f2b2f264640 1 --2- 172.21.15.163:0/3862201859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280d3640 0x7f2b2814d9b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2b18035040 tx=0x7f2b180046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.204+0000 7f2b2f264640 1 -- 172.21.15.163:0/3862201859 shutdown_connections 2024-05-08T06:37:17.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.204+0000 7f2b2f264640 1 --2- 172.21.15.163:0/3862201859 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f2b140762d0 0x7f2b14078790 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.204+0000 7f2b2f264640 1 --2- 172.21.15.163:0/3862201859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b2815f410 0x7f2b280abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.204+0000 7f2b2f264640 1 --2- 172.21.15.163:0/3862201859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280d3640 0x7f2b2814d9b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.204+0000 7f2b2f264640 1 --2- 172.21.15.163:0/3862201859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b280d0040 0x7f2b2814e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.204+0000 7f2b2f264640 1 -- 172.21.15.163:0/3862201859 >> 172.21.15.163:0/3862201859 conn(0x7f2b28073e40 msgr2=0x7f2b280cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:17.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.204+0000 7f2b2f264640 1 -- 172.21.15.163:0/3862201859 shutdown_connections 2024-05-08T06:37:17.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.204+0000 7f2b2f264640 1 -- 172.21.15.163:0/3862201859 wait complete. 2024-05-08T06:37:17.225 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-08T06:37:17.326 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.327+0000 7f6be7b2b640 1 Processor -- start 2024-05-08T06:37:17.326 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.327+0000 7f6be7b2b640 1 -- start start 2024-05-08T06:37:17.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be7b2b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be0162ea0 0x7f6be00b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:17.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be7b2b640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be006e0e0 con 0x7f6be00b6910 2024-05-08T06:37:17.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be7b2b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be006e3c0 con 0x7f6be0148e20 2024-05-08T06:37:17.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be7b2b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6be006e6a0 con 0x7f6be0162ea0 2024-05-08T06:37:17.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be60c8640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6be00b6910 0x7f6be015fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60612/0 (socket says 172.21.15.163:60612) 2024-05-08T06:37:17.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be60c8640 1 -- 172.21.15.163:0/4181794954 learned_addr learned my addr 172.21.15.163:0/4181794954 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:17.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be50c6640 1 --2- 172.21.15.163:0/4181794954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be0162ea0 0x7f6be00b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:17.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be48c5640 1 -- 172.21.15.163:0/4181794954 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2415853422 0 0) 0x7f6be006e0e0 con 0x7f6be00b6910 2024-05-08T06:37:17.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be48c5640 1 -- 172.21.15.163:0/4181794954 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bcc003520 con 0x7f6be00b6910 2024-05-08T06:37:17.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be48c5640 1 -- 172.21.15.163:0/4181794954 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2597594155 0 0) 0x7f6bcc003520 con 0x7f6be00b6910 2024-05-08T06:37:17.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be48c5640 1 -- 172.21.15.163:0/4181794954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be0162ea0 msgr2=0x7f6be00b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:17.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be48c5640 1 --2- 172.21.15.163:0/4181794954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be0162ea0 0x7f6be00b63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be48c5640 1 -- 172.21.15.163:0/4181794954 >> v1:172.21.15.192:6789/0 conn(0x7f6be0148e20 legacy=0x7f6be006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be48c5640 1 -- 172.21.15.163:0/4181794954 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be006e980 con 0x7f6be00b6910 2024-05-08T06:37:17.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be48c5640 1 -- 172.21.15.163:0/4181794954 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f6bd8001a70 con 0x7f6be00b6910 2024-05-08T06:37:17.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.328+0000 7f6be48c5640 1 -- 172.21.15.163:0/4181794954 >> v1:172.21.15.163:6789/0 conn(0x7f6be00b6910 legacy=0x7f6be015fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be48c5640 1 --2- 172.21.15.163:0/4181794954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be0162ea0 0x7f6bcc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:17.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be48c5640 1 --2- 172.21.15.163:0/4181794954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bcc003f70 0x7f6bcc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:17.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be48c5640 1 --2- 172.21.15.163:0/4181794954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bcc004d40 0x7f6bcc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:17.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be48c5640 1 -- 172.21.15.163:0/4181794954 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6be006e6a0 con 0x7f6bcc004d40 2024-05-08T06:37:17.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be48c5640 1 -- 172.21.15.163:0/4181794954 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc00bad0 con 0x7f6bcc003f70 2024-05-08T06:37:17.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be48c5640 1 -- 172.21.15.163:0/4181794954 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6bcc00bcd0 con 0x7f6be0162ea0 2024-05-08T06:37:17.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be60c8640 1 --2- 172.21.15.163:0/4181794954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bcc004d40 0x7f6bcc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:17.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be58c7640 1 --2- 172.21.15.163:0/4181794954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be0162ea0 0x7f6bcc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:17.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be50c6640 1 --2- 172.21.15.163:0/4181794954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bcc003f70 0x7f6bcc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:17.332 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be60c8640 1 -- 172.21.15.163:0/4181794954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be0162ea0 msgr2=0x7f6bcc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:17.332 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be60c8640 1 --2- 172.21.15.163:0/4181794954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be0162ea0 0x7f6bcc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.332 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be60c8640 1 -- 172.21.15.163:0/4181794954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bcc003f70 msgr2=0x7f6bcc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:17.332 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be60c8640 1 --2- 172.21.15.163:0/4181794954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bcc003f70 0x7f6bcc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.332 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be60c8640 1 -- 172.21.15.163:0/4181794954 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc003520 con 0x7f6bcc004d40 2024-05-08T06:37:17.332 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.329+0000 7f6be50c6640 1 --2- 172.21.15.163:0/4181794954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bcc003f70 0x7f6bcc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:17.332 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.330+0000 7f6be60c8640 1 --2- 172.21.15.163:0/4181794954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bcc004d40 0x7f6bcc0051c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800a7a0 tx=0x7f6bcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de13fd426c7fa4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:17.332 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.330+0000 7f6be48c5640 1 -- 172.21.15.163:0/4181794954 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800a3f0 con 0x7f6bcc004d40 2024-05-08T06:37:17.332 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.330+0000 7f6be48c5640 1 -- 172.21.15.163:0/4181794954 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd802f070 con 0x7f6bcc004d40 2024-05-08T06:37:17.332 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.330+0000 7f6be48c5640 1 -- 172.21.15.163:0/4181794954 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd802bb40 con 0x7f6bcc004d40 2024-05-08T06:37:17.332 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.330+0000 7f6be7b2b640 1 -- 172.21.15.163:0/4181794954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bcc004d40 msgr2=0x7f6bcc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.332 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.330+0000 7f6be7b2b640 1 --2- 172.21.15.163:0/4181794954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bcc004d40 0x7f6bcc0051c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800a7a0 tx=0x7f6bcc005830 comp rx=0 tx=0).stop 2024-05-08T06:37:17.332 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.330+0000 7f6be60c8640 1 -- 172.21.15.163:0/4181794954 reap_dead start 2024-05-08T06:37:17.333 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.330+0000 7f6be7b2b640 1 -- 172.21.15.163:0/4181794954 shutdown_connections 2024-05-08T06:37:17.333 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.330+0000 7f6be7b2b640 1 -- 172.21.15.163:0/4181794954 >> 172.21.15.163:0/4181794954 conn(0x7f6be00706c0 msgr2=0x7f6be0070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:17.333 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.330+0000 7f6be7b2b640 1 -- 172.21.15.163:0/4181794954 shutdown_connections 2024-05-08T06:37:17.333 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.330+0000 7f6be7b2b640 1 -- 172.21.15.163:0/4181794954 wait complete. 2024-05-08T06:37:17.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.331+0000 7f6be7b2b640 1 Processor -- start 2024-05-08T06:37:17.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.332+0000 7f6be7b2b640 1 -- start start 2024-05-08T06:37:17.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.332+0000 7f6be7b2b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bcc004d40 0x7f6be0143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:17.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.332+0000 7f6be7b2b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be01473d0 0x7f6be0147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:17.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.332+0000 7f6be7b2b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6be00b6d60 0x7f6be00b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:17.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.332+0000 7f6be7b2b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6be00b6760 con 0x7f6bcc004d40 2024-05-08T06:37:17.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.332+0000 7f6be7b2b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6be00b65e0 con 0x7f6be00b6d60 2024-05-08T06:37:17.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.332+0000 7f6be7b2b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6be01603c0 con 0x7f6be01473d0 2024-05-08T06:37:17.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.332+0000 7f6be58c7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bcc004d40 0x7f6be0143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:17.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.332+0000 7f6be58c7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bcc004d40 0x7f6be0143eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40716/0 (socket says 172.21.15.163:40716) 2024-05-08T06:37:17.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.332+0000 7f6be58c7640 1 -- 172.21.15.163:0/2590487312 learned_addr learned my addr 172.21.15.163:0/2590487312 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:17.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6be60c8640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6be00b6d60 0x7f6be00b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:17.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6be58c7640 1 -- 172.21.15.163:0/2590487312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be01473d0 msgr2=0x7f6be0147850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6be50c6640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be01473d0 0x7f6be0147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:17.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6be58c7640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be01473d0 0x7f6be0147850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6be58c7640 1 -- 172.21.15.163:0/2590487312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6be00b6d60 msgr2=0x7f6be00b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6be58c7640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6be00b6d60 0x7f6be00b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6be58c7640 1 -- 172.21.15.163:0/2590487312 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be01443f0 con 0x7f6bcc004d40 2024-05-08T06:37:17.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6be60c8640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6be00b6d60 0x7f6be00b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:17.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6be50c6640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be01473d0 0x7f6be0147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:17.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6be58c7640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bcc004d40 0x7f6be0143eb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00f7e0 tx=0x7f6bdc00fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:17.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6bd67fc640 1 -- 172.21.15.163:0/2590487312 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0052e0 con 0x7f6bcc004d40 2024-05-08T06:37:17.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6bd67fc640 1 -- 172.21.15.163:0/2590487312 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bdc018070 con 0x7f6bcc004d40 2024-05-08T06:37:17.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6be7b2b640 1 -- 172.21.15.163:0/2590487312 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be00cfc40 con 0x7f6bcc004d40 2024-05-08T06:37:17.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6bd67fc640 1 -- 172.21.15.163:0/2590487312 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00d970 con 0x7f6bcc004d40 2024-05-08T06:37:17.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.333+0000 7f6be7b2b640 1 -- 172.21.15.163:0/2590487312 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be00d0200 con 0x7f6bcc004d40 2024-05-08T06:37:17.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.334+0000 7f6be7b2b640 1 -- 172.21.15.163:0/2590487312 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be006a550 con 0x7f6bcc004d40 2024-05-08T06:37:17.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.335+0000 7f6bd67fc640 1 -- 172.21.15.163:0/2590487312 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6bdc00db10 con 0x7f6bcc004d40 2024-05-08T06:37:17.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.335+0000 7f6bd67fc640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6bc40762d0 0x7f6bc4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:17.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.335+0000 7f6bd67fc640 1 -- 172.21.15.163:0/2590487312 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f6bdc097280 con 0x7f6bcc004d40 2024-05-08T06:37:17.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.335+0000 7f6be50c6640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6bc40762d0 0x7f6bc4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:17.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.336+0000 7f6be50c6640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6bc40762d0 0x7f6bc4078790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6be0148270 tx=0x7f6bd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:17.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.339+0000 7f6bd67fc640 1 -- 172.21.15.163:0/2590487312 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6bdc060fc0 con 0x7f6bcc004d40 2024-05-08T06:37:17.592 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.593+0000 7f6be7b2b640 1 -- 172.21.15.163:0/2590487312 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f6be00d3d60 con 0x7f6bcc004d40 2024-05-08T06:37:17.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.595+0000 7f6bd67fc640 1 -- 172.21.15.163:0/2590487312 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f6bdc0608c0 con 0x7f6bcc004d40 2024-05-08T06:37:17.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.598+0000 7f6be7b2b640 1 -- 172.21.15.163:0/2590487312 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6bc40762d0 msgr2=0x7f6bc4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.598+0000 7f6be7b2b640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6bc40762d0 0x7f6bc4078790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6be0148270 tx=0x7f6bd0008040 comp rx=0 tx=0).stop 2024-05-08T06:37:17.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.598+0000 7f6be7b2b640 1 -- 172.21.15.163:0/2590487312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bcc004d40 msgr2=0x7f6be0143eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.598+0000 7f6be7b2b640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bcc004d40 0x7f6be0143eb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00f7e0 tx=0x7f6bdc00fca0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.598+0000 7f6be7b2b640 1 -- 172.21.15.163:0/2590487312 shutdown_connections 2024-05-08T06:37:17.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.598+0000 7f6be7b2b640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6bc40762d0 0x7f6bc4078790 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.598+0000 7f6be7b2b640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6be00b6d60 0x7f6be00b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.598+0000 7f6be7b2b640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be01473d0 0x7f6be0147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.598+0000 7f6be7b2b640 1 --2- 172.21.15.163:0/2590487312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bcc004d40 0x7f6be0143eb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.598+0000 7f6be7b2b640 1 -- 172.21.15.163:0/2590487312 >> 172.21.15.163:0/2590487312 conn(0x7f6be00706c0 msgr2=0x7f6be00b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:17.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.598+0000 7f6be7b2b640 1 -- 172.21.15.163:0/2590487312 shutdown_connections 2024-05-08T06:37:17.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.598+0000 7f6be7b2b640 1 -- 172.21.15.163:0/2590487312 wait complete. 2024-05-08T06:37:17.608 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-08T06:37:17.709 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.710+0000 7f89a42dc640 1 Processor -- start 2024-05-08T06:37:17.709 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.710+0000 7f89a42dc640 1 -- start start 2024-05-08T06:37:17.710 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.711+0000 7f89a42dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f899c0b6ac0 0x7f899c16a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:17.710 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.711+0000 7f89a42dc640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f899c162fd0 con 0x7f899c0d36b0 2024-05-08T06:37:17.710 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.711+0000 7f89a42dc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f899c1632b0 con 0x7f899c148d90 2024-05-08T06:37:17.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.711+0000 7f89a42dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f899c0b7070 con 0x7f899c0b6ac0 2024-05-08T06:37:17.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.711+0000 7f89a1877640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f899c0d36b0 0x7f899c0b63b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60618/0 (socket says 172.21.15.163:60618) 2024-05-08T06:37:17.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.711+0000 7f89a1877640 1 -- 172.21.15.163:0/282738091 learned_addr learned my addr 172.21.15.163:0/282738091 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:17.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.711+0000 7f89a2879640 1 --2- 172.21.15.163:0/282738091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f899c0b6ac0 0x7f899c16a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:17.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.711+0000 7f89a1076640 1 -- 172.21.15.163:0/282738091 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 711870848 0 0) 0x7f899c162fd0 con 0x7f899c0d36b0 2024-05-08T06:37:17.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.711+0000 7f89a1076640 1 -- 172.21.15.163:0/282738091 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8988003520 con 0x7f899c0d36b0 2024-05-08T06:37:17.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.711+0000 7f89a2879640 1 -- 172.21.15.163:0/282738091 >> v1:172.21.15.192:6789/0 conn(0x7f899c148d90 legacy=0x7f899c0d2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.711+0000 7f89a2879640 1 -- 172.21.15.163:0/282738091 >> v1:172.21.15.163:6789/0 conn(0x7f899c0d36b0 legacy=0x7f899c0b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.711+0000 7f89a2879640 1 -- 172.21.15.163:0/282738091 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899c163590 con 0x7f899c0b6ac0 2024-05-08T06:37:17.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.712+0000 7f89a2879640 1 --2- 172.21.15.163:0/282738091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f899c0b6ac0 0x7f899c16a230 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8994009dc0 tx=0x7f899402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa958c6af6192b1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:17.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.712+0000 7f89a1076640 1 -- 172.21.15.163:0/282738091 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899402ae60 con 0x7f899c0b6ac0 2024-05-08T06:37:17.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.712+0000 7f89a1076640 1 -- 172.21.15.163:0/282738091 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8994007740 con 0x7f899c0b6ac0 2024-05-08T06:37:17.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.712+0000 7f89a1076640 1 -- 172.21.15.163:0/282738091 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994007b10 con 0x7f899c0b6ac0 2024-05-08T06:37:17.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.713+0000 7f89a42dc640 1 -- 172.21.15.163:0/282738091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f899c0b6ac0 msgr2=0x7f899c16a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.713+0000 7f89a42dc640 1 --2- 172.21.15.163:0/282738091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f899c0b6ac0 0x7f899c16a230 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8994009dc0 tx=0x7f899402a640 comp rx=0 tx=0).stop 2024-05-08T06:37:17.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.713+0000 7f89a42dc640 1 -- 172.21.15.163:0/282738091 shutdown_connections 2024-05-08T06:37:17.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.713+0000 7f89a42dc640 1 --2- 172.21.15.163:0/282738091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f899c0b6ac0 0x7f899c16a230 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.713+0000 7f89a42dc640 1 -- 172.21.15.163:0/282738091 >> 172.21.15.163:0/282738091 conn(0x7f899c0b4da0 msgr2=0x7f899c0b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:17.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.713+0000 7f89a42dc640 1 -- 172.21.15.163:0/282738091 shutdown_connections 2024-05-08T06:37:17.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.713+0000 7f89a42dc640 1 -- 172.21.15.163:0/282738091 wait complete. 2024-05-08T06:37:17.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.714+0000 7f89a42dc640 1 Processor -- start 2024-05-08T06:37:17.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.714+0000 7f89a42dc640 1 -- start start 2024-05-08T06:37:17.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a42dc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f899c0b6ac0 0x7f899c13e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:17.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a42dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f899c0d36b0 0x7f899c145160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:17.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a42dc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f899c148d90 0x7f899c147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:17.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a42dc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f899c06e910 con 0x7f899c148d90 2024-05-08T06:37:17.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a42dc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f899c163d90 con 0x7f899c0b6ac0 2024-05-08T06:37:17.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a42dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f899c16a770 con 0x7f899c0d36b0 2024-05-08T06:37:17.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a2879640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f899c148d90 0x7f899c147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:17.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a2879640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f899c148d90 0x7f899c147850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40728/0 (socket says 172.21.15.163:40728) 2024-05-08T06:37:17.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a2879640 1 -- 172.21.15.163:0/1702569911 learned_addr learned my addr 172.21.15.163:0/1702569911 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:17.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a1877640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f899c0d36b0 0x7f899c145160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:17.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a2078640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f899c0b6ac0 0x7f899c13e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:17.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a2879640 1 -- 172.21.15.163:0/1702569911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f899c0d36b0 msgr2=0x7f899c145160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a2879640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f899c0d36b0 0x7f899c145160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a2879640 1 -- 172.21.15.163:0/1702569911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f899c0b6ac0 msgr2=0x7f899c13e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a2879640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f899c0b6ac0 0x7f899c13e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a2879640 1 -- 172.21.15.163:0/1702569911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899c0d0900 con 0x7f899c148d90 2024-05-08T06:37:17.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a2078640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f899c0b6ac0 0x7f899c13e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:17.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.715+0000 7f89a1877640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f899c0d36b0 0x7f899c145160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:17.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.716+0000 7f89a2879640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f899c148d90 0x7f899c147850 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8994009ee0 tx=0x7f8994004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:17.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.716+0000 7f898effd640 1 -- 172.21.15.163:0/1702569911 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994040070 con 0x7f899c148d90 2024-05-08T06:37:17.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.716+0000 7f898effd640 1 -- 172.21.15.163:0/1702569911 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f899402ae60 con 0x7f899c148d90 2024-05-08T06:37:17.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.716+0000 7f898effd640 1 -- 172.21.15.163:0/1702569911 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994031be0 con 0x7f899c148d90 2024-05-08T06:37:17.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.716+0000 7f89a42dc640 1 -- 172.21.15.163:0/1702569911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899c145c70 con 0x7f899c148d90 2024-05-08T06:37:17.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.716+0000 7f89a42dc640 1 -- 172.21.15.163:0/1702569911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899c146230 con 0x7f899c148d90 2024-05-08T06:37:17.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.717+0000 7f89a42dc640 1 -- 172.21.15.163:0/1702569911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899c13ee40 con 0x7f899c148d90 2024-05-08T06:37:17.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.717+0000 7f898effd640 1 -- 172.21.15.163:0/1702569911 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8994005d10 con 0x7f899c148d90 2024-05-08T06:37:17.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.718+0000 7f898effd640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f89800762d0 0x7f8980078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:17.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.718+0000 7f898effd640 1 -- 172.21.15.163:0/1702569911 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f89940b8c60 con 0x7f899c148d90 2024-05-08T06:37:17.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.718+0000 7f89a2078640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f89800762d0 0x7f8980078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:17.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.718+0000 7f89a2078640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f89800762d0 0x7f8980078790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8990007fd0 tx=0x7f8990009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:17.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.722+0000 7f898effd640 1 -- 172.21.15.163:0/1702569911 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f899c13ee40 con 0x7f899c148d90 2024-05-08T06:37:17.972 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.973+0000 7f89a42dc640 1 -- 172.21.15.163:0/1702569911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f899c06edd0 con 0x7f899c148d90 2024-05-08T06:37:17.974 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.975+0000 7f898effd640 1 -- 172.21.15.163:0/1702569911 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f8994082920 con 0x7f899c148d90 2024-05-08T06:37:17.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.977+0000 7f89a42dc640 1 -- 172.21.15.163:0/1702569911 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f89800762d0 msgr2=0x7f8980078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.977+0000 7f89a42dc640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f89800762d0 0x7f8980078790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8990007fd0 tx=0x7f8990009040 comp rx=0 tx=0).stop 2024-05-08T06:37:17.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.977+0000 7f89a42dc640 1 -- 172.21.15.163:0/1702569911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f899c148d90 msgr2=0x7f899c147850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:17.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.977+0000 7f89a42dc640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f899c148d90 0x7f899c147850 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8994009ee0 tx=0x7f8994004a40 comp rx=0 tx=0).stop 2024-05-08T06:37:17.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.977+0000 7f89a42dc640 1 -- 172.21.15.163:0/1702569911 shutdown_connections 2024-05-08T06:37:17.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.977+0000 7f89a42dc640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f89800762d0 0x7f8980078790 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.977+0000 7f89a42dc640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f899c148d90 0x7f899c147850 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.977+0000 7f89a42dc640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f899c0d36b0 0x7f899c145160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.977+0000 7f89a42dc640 1 --2- 172.21.15.163:0/1702569911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f899c0b6ac0 0x7f899c13e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:17.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.977+0000 7f89a42dc640 1 -- 172.21.15.163:0/1702569911 >> 172.21.15.163:0/1702569911 conn(0x7f899c0b4da0 msgr2=0x7f899c0b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:17.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.978+0000 7f89a42dc640 1 -- 172.21.15.163:0/1702569911 shutdown_connections 2024-05-08T06:37:17.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:17.978+0000 7f89a42dc640 1 -- 172.21.15.163:0/1702569911 wait complete. 2024-05-08T06:37:17.987 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-08T06:37:18.087 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.088+0000 7f6fdfb89640 1 Processor -- start 2024-05-08T06:37:18.087 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.088+0000 7f6fdfb89640 1 -- start start 2024-05-08T06:37:18.089 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.089+0000 7f6fdfb89640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd81607d0 0x7f6fd80b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.089 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.089+0000 7f6fdfb89640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6fd80d1270 0x7f6fd80d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.089 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.089+0000 7f6fdfb89640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd815d580 con 0x7f6fd81603f0 2024-05-08T06:37:18.089 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.089+0000 7f6fdfb89640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f6fd80b4e70 con 0x7f6fd80d1270 2024-05-08T06:37:18.089 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.089+0000 7f6fdfb89640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd80b5150 con 0x7f6fd81607d0 2024-05-08T06:37:18.089 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.089+0000 7f6fde126640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6fd80d1270 0x7f6fd80d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.089 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.089+0000 7f6fdd925640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6fd81603f0 0x7f6fd815ce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60632/0 (socket says 172.21.15.163:60632) 2024-05-08T06:37:18.090 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.089+0000 7f6fdd925640 1 -- 172.21.15.163:0/2180839544 learned_addr learned my addr 172.21.15.163:0/2180839544 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:18.090 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.090+0000 7f6fdd124640 1 --2- 172.21.15.163:0/2180839544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd81607d0 0x7f6fd80b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.090 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.090+0000 7f6fde126640 1 -- 172.21.15.163:0/2180839544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd81607d0 msgr2=0x7f6fd80b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:18.090 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.090+0000 7f6fde126640 1 --2- 172.21.15.163:0/2180839544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd81607d0 0x7f6fd80b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.090 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.090+0000 7f6fde126640 1 -- 172.21.15.163:0/2180839544 >> v1:172.21.15.163:6789/0 conn(0x7f6fd81603f0 legacy=0x7f6fd815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.090 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.090+0000 7f6fdc923640 1 -- 172.21.15.163:0/2180839544 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1856906500 0 0) 0x7f6fd815d580 con 0x7f6fd81603f0 2024-05-08T06:37:18.090 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.090+0000 7f6fde126640 1 -- 172.21.15.163:0/2180839544 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd80b5430 con 0x7f6fd80d1270 2024-05-08T06:37:18.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.090+0000 7f6fde126640 1 --2- 172.21.15.163:0/2180839544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fd80d1270 0x7f6fd80d1700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009d90 tx=0x7f6fd002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69a747eb24626be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.091+0000 7f6fdc923640 1 -- 172.21.15.163:0/2180839544 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0007740 con 0x7f6fd80d1270 2024-05-08T06:37:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.091+0000 7f6fdc923640 1 -- 172.21.15.163:0/2180839544 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fd00078e0 con 0x7f6fd80d1270 2024-05-08T06:37:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.091+0000 7f6fdc923640 1 -- 172.21.15.163:0/2180839544 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0007cb0 con 0x7f6fd80d1270 2024-05-08T06:37:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.091+0000 7f6fdfb89640 1 -- 172.21.15.163:0/2180839544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fd80d1270 msgr2=0x7f6fd80d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.091+0000 7f6fdfb89640 1 --2- 172.21.15.163:0/2180839544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fd80d1270 0x7f6fd80d1700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009d90 tx=0x7f6fd002a660 comp rx=0 tx=0).stop 2024-05-08T06:37:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.091+0000 7f6fdfb89640 1 -- 172.21.15.163:0/2180839544 shutdown_connections 2024-05-08T06:37:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.091+0000 7f6fdfb89640 1 --2- 172.21.15.163:0/2180839544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fd80d1270 0x7f6fd80d1700 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.091+0000 7f6fdfb89640 1 --2- 172.21.15.163:0/2180839544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd81607d0 0x7f6fd80b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.091+0000 7f6fdfb89640 1 -- 172.21.15.163:0/2180839544 >> 172.21.15.163:0/2180839544 conn(0x7f6fd806c500 msgr2=0x7f6fd806c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.091+0000 7f6fdfb89640 1 -- 172.21.15.163:0/2180839544 shutdown_connections 2024-05-08T06:37:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.091+0000 7f6fdfb89640 1 -- 172.21.15.163:0/2180839544 wait complete. 2024-05-08T06:37:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.092+0000 7f6fdfb89640 1 Processor -- start 2024-05-08T06:37:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.092+0000 7f6fdfb89640 1 -- start start 2024-05-08T06:37:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.093+0000 7f6fdfb89640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd80d1270 0x7f6fd81f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.093+0000 7f6fdfb89640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fd81603f0 0x7f6fd8151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.093+0000 7f6fdfb89640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fd81607d0 0x7f6fd8152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.093+0000 7f6fdfb89640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd8072840 con 0x7f6fd81607d0 2024-05-08T06:37:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.093+0000 7f6fdfb89640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd80b59f0 con 0x7f6fd81603f0 2024-05-08T06:37:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.093+0000 7f6fdfb89640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd80729c0 con 0x7f6fd80d1270 2024-05-08T06:37:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.093+0000 7f6fde126640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fd81607d0 0x7f6fd8152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.093+0000 7f6fde126640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fd81607d0 0x7f6fd8152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40740/0 (socket says 172.21.15.163:40740) 2024-05-08T06:37:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.093+0000 7f6fde126640 1 -- 172.21.15.163:0/1069002658 learned_addr learned my addr 172.21.15.163:0/1069002658 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.093+0000 7f6fdd925640 1 --2- 172.21.15.163:0/1069002658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd80d1270 0x7f6fd81f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.093+0000 7f6fdd124640 1 --2- 172.21.15.163:0/1069002658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fd81603f0 0x7f6fd8151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.094+0000 7f6fde126640 1 -- 172.21.15.163:0/1069002658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd80d1270 msgr2=0x7f6fd81f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.094+0000 7f6fde126640 1 --2- 172.21.15.163:0/1069002658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd80d1270 0x7f6fd81f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.094+0000 7f6fde126640 1 -- 172.21.15.163:0/1069002658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fd81603f0 msgr2=0x7f6fd8151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.094+0000 7f6fde126640 1 --2- 172.21.15.163:0/1069002658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fd81603f0 0x7f6fd8151450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.094+0000 7f6fde126640 1 -- 172.21.15.163:0/1069002658 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd8151ca0 con 0x7f6fd81607d0 2024-05-08T06:37:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.094+0000 7f6fde126640 1 --2- 172.21.15.163:0/1069002658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fd81607d0 0x7f6fd8152d40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009eb0 tx=0x7f6fd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.094+0000 7f6fce7fc640 1 -- 172.21.15.163:0/1069002658 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0040070 con 0x7f6fd81607d0 2024-05-08T06:37:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.094+0000 7f6fce7fc640 1 -- 172.21.15.163:0/1069002658 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fd0031c40 con 0x7f6fd81607d0 2024-05-08T06:37:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.094+0000 7f6fdfb89640 1 -- 172.21.15.163:0/1069002658 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd8151f80 con 0x7f6fd81607d0 2024-05-08T06:37:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.094+0000 7f6fce7fc640 1 -- 172.21.15.163:0/1069002658 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0009450 con 0x7f6fd81607d0 2024-05-08T06:37:18.097 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.094+0000 7f6fdfb89640 1 -- 172.21.15.163:0/1069002658 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd8152540 con 0x7f6fd81607d0 2024-05-08T06:37:18.097 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.095+0000 7f6fdfb89640 1 -- 172.21.15.163:0/1069002658 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd806a550 con 0x7f6fd81607d0 2024-05-08T06:37:18.097 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.096+0000 7f6fce7fc640 1 -- 172.21.15.163:0/1069002658 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6fd0039040 con 0x7f6fd81607d0 2024-05-08T06:37:18.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.096+0000 7f6fce7fc640 1 --2- 172.21.15.163:0/1069002658 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6fbc0762d0 0x7f6fbc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.096+0000 7f6fce7fc640 1 -- 172.21.15.163:0/1069002658 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f6fd00b8160 con 0x7f6fd81607d0 2024-05-08T06:37:18.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.096+0000 7f6fdd925640 1 --2- 172.21.15.163:0/1069002658 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6fbc0762d0 0x7f6fbc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.097+0000 7f6fdd925640 1 --2- 172.21.15.163:0/1069002658 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6fbc0762d0 0x7f6fbc078790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4000f60 tx=0x7f6fd4007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:18.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.100+0000 7f6fce7fc640 1 -- 172.21.15.163:0/1069002658 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6fd0081730 con 0x7f6fd81607d0 2024-05-08T06:37:18.352 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.353+0000 7f6fdfb89640 1 -- 172.21.15.163:0/1069002658 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f6fd80b18b0 con 0x7f6fd81607d0 2024-05-08T06:37:18.353 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.355+0000 7f6fce7fc640 1 -- 172.21.15.163:0/1069002658 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f6fd0085a30 con 0x7f6fd81607d0 2024-05-08T06:37:18.356 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.357+0000 7f6fdfb89640 1 -- 172.21.15.163:0/1069002658 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6fbc0762d0 msgr2=0x7f6fbc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.356 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.357+0000 7f6fdfb89640 1 --2- 172.21.15.163:0/1069002658 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6fbc0762d0 0x7f6fbc078790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4000f60 tx=0x7f6fd4007cf0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.356 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.357+0000 7f6fdfb89640 1 -- 172.21.15.163:0/1069002658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fd81607d0 msgr2=0x7f6fd8152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.356 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.357+0000 7f6fdfb89640 1 --2- 172.21.15.163:0/1069002658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fd81607d0 0x7f6fd8152d40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009eb0 tx=0x7f6fd0004a40 comp rx=0 tx=0).stop 2024-05-08T06:37:18.356 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.357+0000 7f6fdfb89640 1 -- 172.21.15.163:0/1069002658 shutdown_connections 2024-05-08T06:37:18.356 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.357+0000 7f6fdfb89640 1 --2- 172.21.15.163:0/1069002658 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6fbc0762d0 0x7f6fbc078790 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.357 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.357+0000 7f6fdfb89640 1 --2- 172.21.15.163:0/1069002658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fd81607d0 0x7f6fd8152d40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.357 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.357+0000 7f6fdfb89640 1 --2- 172.21.15.163:0/1069002658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fd81603f0 0x7f6fd8151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.357 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.357+0000 7f6fdfb89640 1 --2- 172.21.15.163:0/1069002658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd80d1270 0x7f6fd81f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.357 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.357+0000 7f6fdfb89640 1 -- 172.21.15.163:0/1069002658 >> 172.21.15.163:0/1069002658 conn(0x7f6fd806c500 msgr2=0x7f6fd80b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:18.357 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.357+0000 7f6fdfb89640 1 -- 172.21.15.163:0/1069002658 shutdown_connections 2024-05-08T06:37:18.357 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.357+0000 7f6fdfb89640 1 -- 172.21.15.163:0/1069002658 wait complete. 2024-05-08T06:37:18.367 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-08T06:37:18.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.469+0000 7f8065075640 1 Processor -- start 2024-05-08T06:37:18.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.469+0000 7f8065075640 1 -- start start 2024-05-08T06:37:18.469 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.470+0000 7f8065075640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f806015f5a0 0x7f80600d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.470 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.470+0000 7f8065075640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f80600d19c0 0x7f80600d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.470 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.470+0000 7f8065075640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8060163250 0x7f80601636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.470 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.470+0000 7f8065075640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f8060163f00 con 0x7f80600d19c0 2024-05-08T06:37:18.470 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.470+0000 7f8065075640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f80600cfe20 con 0x7f8060163250 2024-05-08T06:37:18.470 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.470+0000 7f8065075640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f80600d0100 con 0x7f806015f5a0 2024-05-08T06:37:18.470 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.470+0000 7f805e59c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f80600d19c0 0x7f80600d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.470 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.470+0000 7f805e59c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f80600d19c0 0x7f80600d1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40754/0 (socket says 172.21.15.163:40754) 2024-05-08T06:37:18.470 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.470+0000 7f805e59c640 1 -- 172.21.15.163:0/2926185582 learned_addr learned my addr 172.21.15.163:0/2926185582 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:18.470 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.470+0000 7f805e59c640 1 -- 172.21.15.163:0/2926185582 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f806015f5a0 msgr2=0x7f80600d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:18.470 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.471+0000 7f805e59c640 1 --2- 172.21.15.163:0/2926185582 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f806015f5a0 0x7f80600d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.470 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.471+0000 7f805e59c640 1 -- 172.21.15.163:0/2926185582 >> v2:172.21.15.192:3300/0 conn(0x7f8060163250 msgr2=0x7f80601636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:18.470 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.471+0000 7f805e59c640 1 --2- 172.21.15.163:0/2926185582 >> v2:172.21.15.192:3300/0 conn(0x7f8060163250 0x7f80601636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.471+0000 7f805e59c640 1 -- 172.21.15.163:0/2926185582 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80600d03e0 con 0x7f80600d19c0 2024-05-08T06:37:18.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.471+0000 7f805e59c640 1 --2- 172.21.15.163:0/2926185582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80600d19c0 0x7f80600d1e00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80600b8170 tx=0x7f805402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207d636b0e1f228e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:18.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.471+0000 7f805dd9b640 1 -- 172.21.15.163:0/2926185582 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054037070 con 0x7f80600d19c0 2024-05-08T06:37:18.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.471+0000 7f805dd9b640 1 -- 172.21.15.163:0/2926185582 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8054032440 con 0x7f80600d19c0 2024-05-08T06:37:18.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.471+0000 7f805dd9b640 1 -- 172.21.15.163:0/2926185582 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054032810 con 0x7f80600d19c0 2024-05-08T06:37:18.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.472+0000 7f8065075640 1 -- 172.21.15.163:0/2926185582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80600d19c0 msgr2=0x7f80600d1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.472+0000 7f8065075640 1 --2- 172.21.15.163:0/2926185582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80600d19c0 0x7f80600d1e00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80600b8170 tx=0x7f805402a1e0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.472+0000 7f8065075640 1 -- 172.21.15.163:0/2926185582 shutdown_connections 2024-05-08T06:37:18.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.472+0000 7f8065075640 1 --2- 172.21.15.163:0/2926185582 >> v2:172.21.15.192:3300/0 conn(0x7f8060163250 0x7f80601636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.472+0000 7f8065075640 1 --2- 172.21.15.163:0/2926185582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80600d19c0 0x7f80600d1e00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.472+0000 7f8065075640 1 --2- 172.21.15.163:0/2926185582 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f806015f5a0 0x7f80600d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.472+0000 7f8065075640 1 -- 172.21.15.163:0/2926185582 >> 172.21.15.163:0/2926185582 conn(0x7f8060074080 msgr2=0x7f806006d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:18.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.472+0000 7f8065075640 1 -- 172.21.15.163:0/2926185582 shutdown_connections 2024-05-08T06:37:18.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.472+0000 7f8065075640 1 -- 172.21.15.163:0/2926185582 wait complete. 2024-05-08T06:37:18.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.474+0000 7f8065075640 1 Processor -- start 2024-05-08T06:37:18.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.474+0000 7f8065075640 1 -- start start 2024-05-08T06:37:18.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.474+0000 7f8065075640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80600d19c0 0x7f80600b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.474+0000 7f8065075640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f806015f5a0 0x7f8060161d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.474+0000 7f8065075640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8060163250 0x7f80600c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.474+0000 7f8065075640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f806015f980 con 0x7f806015f5a0 2024-05-08T06:37:18.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.474+0000 7f8065075640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8060163be0 con 0x7f8060163250 2024-05-08T06:37:18.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.474+0000 7f8065075640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f806015fb00 con 0x7f80600d19c0 2024-05-08T06:37:18.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.474+0000 7f805e59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f806015f5a0 0x7f8060161d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.474+0000 7f805e59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f806015f5a0 0x7f8060161d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40758/0 (socket says 172.21.15.163:40758) 2024-05-08T06:37:18.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.474+0000 7f805e59c640 1 -- 172.21.15.163:0/4248496132 learned_addr learned my addr 172.21.15.163:0/4248496132 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:18.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.475+0000 7f805ed9d640 1 --2- 172.21.15.163:0/4248496132 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80600d19c0 0x7f80600b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.475+0000 7f805f59e640 1 --2- 172.21.15.163:0/4248496132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8060163250 0x7f80600c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.475+0000 7f805e59c640 1 -- 172.21.15.163:0/4248496132 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80600d19c0 msgr2=0x7f80600b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.475+0000 7f805e59c640 1 --2- 172.21.15.163:0/4248496132 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80600d19c0 0x7f80600b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.475+0000 7f805e59c640 1 -- 172.21.15.163:0/4248496132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8060163250 msgr2=0x7f80600c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.475+0000 7f805e59c640 1 --2- 172.21.15.163:0/4248496132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8060163250 0x7f80600c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.475+0000 7f805e59c640 1 -- 172.21.15.163:0/4248496132 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80600b6b50 con 0x7f806015f5a0 2024-05-08T06:37:18.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.475+0000 7f805e59c640 1 --2- 172.21.15.163:0/4248496132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f806015f5a0 0x7f8060161d90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8054000c00 tx=0x7f80540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:18.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.475+0000 7f80477fe640 1 -- 172.21.15.163:0/4248496132 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054041070 con 0x7f806015f5a0 2024-05-08T06:37:18.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.475+0000 7f80477fe640 1 -- 172.21.15.163:0/4248496132 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f805403c870 con 0x7f806015f5a0 2024-05-08T06:37:18.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.475+0000 7f8065075640 1 -- 172.21.15.163:0/4248496132 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80600b6e30 con 0x7f806015f5a0 2024-05-08T06:37:18.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.475+0000 7f80477fe640 1 -- 172.21.15.163:0/4248496132 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054037040 con 0x7f806015f5a0 2024-05-08T06:37:18.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.475+0000 7f8065075640 1 -- 172.21.15.163:0/4248496132 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80600b73f0 con 0x7f806015f5a0 2024-05-08T06:37:18.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.476+0000 7f8065075640 1 -- 172.21.15.163:0/4248496132 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f806006a550 con 0x7f806015f5a0 2024-05-08T06:37:18.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.477+0000 7f80477fe640 1 -- 172.21.15.163:0/4248496132 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8054044070 con 0x7f806015f5a0 2024-05-08T06:37:18.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.477+0000 7f80477fe640 1 --2- 172.21.15.163:0/4248496132 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f804c076300 0x7f804c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.477+0000 7f80477fe640 1 -- 172.21.15.163:0/4248496132 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f80540b7d60 con 0x7f806015f5a0 2024-05-08T06:37:18.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.477+0000 7f805ed9d640 1 --2- 172.21.15.163:0/4248496132 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f804c076300 0x7f804c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.478 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.478+0000 7f805ed9d640 1 --2- 172.21.15.163:0/4248496132 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f804c076300 0x7f804c0787c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f805000a400 tx=0x7f805000d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:18.480 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.482+0000 7f80477fe640 1 -- 172.21.15.163:0/4248496132 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8054081450 con 0x7f806015f5a0 2024-05-08T06:37:18.733 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.734+0000 7f8065075640 1 -- 172.21.15.163:0/4248496132 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f80600d3170 con 0x7f806015f5a0 2024-05-08T06:37:18.737 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.738+0000 7f80477fe640 1 -- 172.21.15.163:0/4248496132 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f8054085750 con 0x7f806015f5a0 2024-05-08T06:37:18.739 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.740+0000 7f8065075640 1 -- 172.21.15.163:0/4248496132 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f804c076300 msgr2=0x7f804c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.739 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.740+0000 7f8065075640 1 --2- 172.21.15.163:0/4248496132 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f804c076300 0x7f804c0787c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f805000a400 tx=0x7f805000d040 comp rx=0 tx=0).stop 2024-05-08T06:37:18.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.740+0000 7f8065075640 1 -- 172.21.15.163:0/4248496132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f806015f5a0 msgr2=0x7f8060161d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.740+0000 7f8065075640 1 --2- 172.21.15.163:0/4248496132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f806015f5a0 0x7f8060161d90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8054000c00 tx=0x7f80540046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.740+0000 7f8065075640 1 -- 172.21.15.163:0/4248496132 shutdown_connections 2024-05-08T06:37:18.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.740+0000 7f8065075640 1 --2- 172.21.15.163:0/4248496132 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f804c076300 0x7f804c0787c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.740+0000 7f8065075640 1 --2- 172.21.15.163:0/4248496132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8060163250 0x7f80600c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.740+0000 7f8065075640 1 --2- 172.21.15.163:0/4248496132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f806015f5a0 0x7f8060161d90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.740+0000 7f8065075640 1 --2- 172.21.15.163:0/4248496132 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80600d19c0 0x7f80600b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.740+0000 7f8065075640 1 -- 172.21.15.163:0/4248496132 >> 172.21.15.163:0/4248496132 conn(0x7f8060074080 msgr2=0x7f806015e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:18.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.740+0000 7f8065075640 1 -- 172.21.15.163:0/4248496132 shutdown_connections 2024-05-08T06:37:18.740 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.740+0000 7f8065075640 1 -- 172.21.15.163:0/4248496132 wait complete. 2024-05-08T06:37:18.750 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-08T06:37:18.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.852+0000 7f9ac0a59640 1 Processor -- start 2024-05-08T06:37:18.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.853+0000 7f9ac0a59640 1 -- start start 2024-05-08T06:37:18.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.853+0000 7f9ac0a59640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9abc0d1270 0x7f9abc0b0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.853+0000 7f9ac0a59640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9abc0ae250 con 0x7f9abc074370 2024-05-08T06:37:18.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.853+0000 7f9ac0a59640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9abc0ae530 con 0x7f9abc06ef90 2024-05-08T06:37:18.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ac0a59640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9abc0ae000 con 0x7f9abc0d1270 2024-05-08T06:37:18.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab9d9b640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f9abc074370 0x7f9abc0d1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60638/0 (socket says 172.21.15.163:60638) 2024-05-08T06:37:18.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab9d9b640 1 -- 172.21.15.163:0/3911608074 learned_addr learned my addr 172.21.15.163:0/3911608074 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:18.856 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9aba59c640 1 --2- 172.21.15.163:0/3911608074 >> v2:172.21.15.192:3300/0 conn(0x7f9abc0d1270 0x7f9abc0b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 -- 172.21.15.163:0/3911608074 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1555523432 0 0) 0x7f9abc0ae250 con 0x7f9abc074370 2024-05-08T06:37:18.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 -- 172.21.15.163:0/3911608074 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9aa4003520 con 0x7f9abc074370 2024-05-08T06:37:18.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 -- 172.21.15.163:0/3911608074 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2347862737 0 0) 0x7f9aa4003520 con 0x7f9abc074370 2024-05-08T06:37:18.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 -- 172.21.15.163:0/3911608074 >> v2:172.21.15.192:3300/0 conn(0x7f9abc0d1270 msgr2=0x7f9abc0b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:18.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 --2- 172.21.15.163:0/3911608074 >> v2:172.21.15.192:3300/0 conn(0x7f9abc0d1270 0x7f9abc0b0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 -- 172.21.15.163:0/3911608074 >> v1:172.21.15.192:6789/0 conn(0x7f9abc06ef90 legacy=0x7f9abc072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 -- 172.21.15.163:0/3911608074 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc06e4b0 con 0x7f9abc074370 2024-05-08T06:37:18.857 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 -- 172.21.15.163:0/3911608074 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f9ab00016f0 con 0x7f9abc074370 2024-05-08T06:37:18.858 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 -- 172.21.15.163:0/3911608074 >> v1:172.21.15.163:6789/0 conn(0x7f9abc074370 legacy=0x7f9abc0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.858 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 --2- 172.21.15.163:0/3911608074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9aa4003a50 0x7f9aa4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.858 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 --2- 172.21.15.163:0/3911608074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9aa4004800 0x7f9aa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.858 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 --2- 172.21.15.163:0/3911608074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa40051c0 0x7f9aa400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.858 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 -- 172.21.15.163:0/3911608074 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc0ae000 con 0x7f9aa4003a50 2024-05-08T06:37:18.858 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 -- 172.21.15.163:0/3911608074 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa400c160 con 0x7f9aa4004800 2024-05-08T06:37:18.858 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab959a640 1 -- 172.21.15.163:0/3911608074 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa400c360 con 0x7f9aa40051c0 2024-05-08T06:37:18.859 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9aba59c640 1 --2- 172.21.15.163:0/3911608074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9aa4003a50 0x7f9aa4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.859 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.854+0000 7f9ab9d9b640 1 --2- 172.21.15.163:0/3911608074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9aa4004800 0x7f9aa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.859 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.855+0000 7f9abad9d640 1 --2- 172.21.15.163:0/3911608074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa40051c0 0x7f9aa400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.859 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.855+0000 7f9aba59c640 1 -- 172.21.15.163:0/3911608074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa40051c0 msgr2=0x7f9aa400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:18.859 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.855+0000 7f9aba59c640 1 --2- 172.21.15.163:0/3911608074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa40051c0 0x7f9aa400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.859 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.855+0000 7f9aba59c640 1 -- 172.21.15.163:0/3911608074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9aa4004800 msgr2=0x7f9aa4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:18.859 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.855+0000 7f9aba59c640 1 --2- 172.21.15.163:0/3911608074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9aa4004800 0x7f9aa4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.860 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.855+0000 7f9aba59c640 1 -- 172.21.15.163:0/3911608074 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa4003520 con 0x7f9aa4003a50 2024-05-08T06:37:18.860 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.855+0000 7f9abad9d640 1 -- 172.21.15.163:0/3911608074 reap_dead start 2024-05-08T06:37:18.860 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.855+0000 7f9aba59c640 1 --2- 172.21.15.163:0/3911608074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9aa4003a50 0x7f9aa4003e70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009ac0 tx=0x7f9aa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87960f207b205784 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:18.860 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.855+0000 7f9ab959a640 1 -- 172.21.15.163:0/3911608074 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac036070 con 0x7f9aa4003a50 2024-05-08T06:37:18.860 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.856+0000 7f9ab959a640 1 -- 172.21.15.163:0/3911608074 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aac031400 con 0x7f9aa4003a50 2024-05-08T06:37:18.860 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.856+0000 7f9ab959a640 1 -- 172.21.15.163:0/3911608074 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac009690 con 0x7f9aa4003a50 2024-05-08T06:37:18.860 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.856+0000 7f9ac0a59640 1 -- 172.21.15.163:0/3911608074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9aa4003a50 msgr2=0x7f9aa4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.856+0000 7f9ac0a59640 1 --2- 172.21.15.163:0/3911608074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9aa4003a50 0x7f9aa4003e70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009ac0 tx=0x7f9aa4005830 comp rx=0 tx=0).stop 2024-05-08T06:37:18.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.856+0000 7f9ac0a59640 1 -- 172.21.15.163:0/3911608074 shutdown_connections 2024-05-08T06:37:18.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.856+0000 7f9ac0a59640 1 --2- 172.21.15.163:0/3911608074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9aa4003a50 0x7f9aa4003e70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.856+0000 7f9ac0a59640 1 -- 172.21.15.163:0/3911608074 >> 172.21.15.163:0/3911608074 conn(0x7f9abc06c500 msgr2=0x7f9abc06c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:18.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.856+0000 7f9ac0a59640 1 -- 172.21.15.163:0/3911608074 shutdown_connections 2024-05-08T06:37:18.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.856+0000 7f9ac0a59640 1 -- 172.21.15.163:0/3911608074 wait complete. 2024-05-08T06:37:18.864 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.857+0000 7f9ac0a59640 1 Processor -- start 2024-05-08T06:37:18.864 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.858+0000 7f9ac0a59640 1 -- start start 2024-05-08T06:37:18.864 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.858+0000 7f9ac0a59640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa4003a50 0x7f9abc159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.864 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.858+0000 7f9ac0a59640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9abc15efb0 0x7f9abc15a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.864 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.858+0000 7f9ac0a59640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9abc1472e0 0x7f9abc0b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.864 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.858+0000 7f9ac0a59640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc06f420 con 0x7f9abc15efb0 2024-05-08T06:37:18.864 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.858+0000 7f9ac0a59640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc06eb00 con 0x7f9abc1472e0 2024-05-08T06:37:18.864 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.858+0000 7f9ac0a59640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9abc15d3d0 con 0x7f9aa4003a50 2024-05-08T06:37:18.864 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.858+0000 7f9ab9d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9abc15efb0 0x7f9abc15a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.858+0000 7f9ab9d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9abc15efb0 0x7f9abc15a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40770/0 (socket says 172.21.15.163:40770) 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.858+0000 7f9ab9d9b640 1 -- 172.21.15.163:0/1693607422 learned_addr learned my addr 172.21.15.163:0/1693607422 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.859+0000 7f9abad9d640 1 --2- 172.21.15.163:0/1693607422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9abc1472e0 0x7f9abc0b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.859+0000 7f9ab9d9b640 1 -- 172.21.15.163:0/1693607422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa4003a50 msgr2=0x7f9abc159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.859+0000 7f9aba59c640 1 --2- 172.21.15.163:0/1693607422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa4003a50 0x7f9abc159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.859+0000 7f9ab9d9b640 1 --2- 172.21.15.163:0/1693607422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa4003a50 0x7f9abc159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.859+0000 7f9ab9d9b640 1 -- 172.21.15.163:0/1693607422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9abc1472e0 msgr2=0x7f9abc0b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.859+0000 7f9ab9d9b640 1 --2- 172.21.15.163:0/1693607422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9abc1472e0 0x7f9abc0b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.859+0000 7f9ab9d9b640 1 -- 172.21.15.163:0/1693607422 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc15ac30 con 0x7f9abc15efb0 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.859+0000 7f9ab9d9b640 1 --2- 172.21.15.163:0/1693607422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9abc15efb0 0x7f9abc15a3c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00072f0 tx=0x7f9ab00077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.859+0000 7f9aaaffd640 1 -- 172.21.15.163:0/1693607422 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000f9c0 con 0x7f9abc15efb0 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.859+0000 7f9aaaffd640 1 -- 172.21.15.163:0/1693607422 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab0018070 con 0x7f9abc15efb0 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.859+0000 7f9ac0a59640 1 -- 172.21.15.163:0/1693607422 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9abc159280 con 0x7f9abc15efb0 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.859+0000 7f9aaaffd640 1 -- 172.21.15.163:0/1693607422 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000fda0 con 0x7f9abc15efb0 2024-05-08T06:37:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.859+0000 7f9ac0a59640 1 -- 172.21.15.163:0/1693607422 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9abc15af10 con 0x7f9abc15efb0 2024-05-08T06:37:18.866 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.860+0000 7f9ac0a59640 1 -- 172.21.15.163:0/1693607422 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9abc06a550 con 0x7f9abc15efb0 2024-05-08T06:37:18.866 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.861+0000 7f9aaaffd640 1 -- 172.21.15.163:0/1693607422 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9ab001d460 con 0x7f9abc15efb0 2024-05-08T06:37:18.866 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.861+0000 7f9aaaffd640 1 --2- 172.21.15.163:0/1693607422 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9a9c0762d0 0x7f9a9c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:18.866 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.861+0000 7f9aba59c640 1 --2- 172.21.15.163:0/1693607422 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9a9c0762d0 0x7f9a9c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:18.866 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.861+0000 7f9aaaffd640 1 -- 172.21.15.163:0/1693607422 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f9ab0097250 con 0x7f9abc15efb0 2024-05-08T06:37:18.866 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.862+0000 7f9aba59c640 1 --2- 172.21.15.163:0/1693607422 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9a9c0762d0 0x7f9a9c078790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9aac033000 tx=0x7f9aac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:18.866 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:18.866+0000 7f9aaaffd640 1 -- 172.21.15.163:0/1693607422 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9ab0060f90 con 0x7f9abc15efb0 2024-05-08T06:37:19.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.120+0000 7f9ac0a59640 1 -- 172.21.15.163:0/1693607422 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f9abc0008d0 con 0x7f9abc15efb0 2024-05-08T06:37:19.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.124+0000 7f9aaaffd640 1 -- 172.21.15.163:0/1693607422 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7f9ab0060890 con 0x7f9abc15efb0 2024-05-08T06:37:19.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.126+0000 7f9ac0a59640 1 -- 172.21.15.163:0/1693607422 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9a9c0762d0 msgr2=0x7f9a9c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.126+0000 7f9ac0a59640 1 --2- 172.21.15.163:0/1693607422 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9a9c0762d0 0x7f9a9c078790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9aac033000 tx=0x7f9aac008030 comp rx=0 tx=0).stop 2024-05-08T06:37:19.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.126+0000 7f9ac0a59640 1 -- 172.21.15.163:0/1693607422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9abc15efb0 msgr2=0x7f9abc15a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.126+0000 7f9ac0a59640 1 --2- 172.21.15.163:0/1693607422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9abc15efb0 0x7f9abc15a3c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00072f0 tx=0x7f9ab00077b0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.126+0000 7f9ac0a59640 1 -- 172.21.15.163:0/1693607422 shutdown_connections 2024-05-08T06:37:19.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.126+0000 7f9ac0a59640 1 --2- 172.21.15.163:0/1693607422 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9a9c0762d0 0x7f9a9c078790 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.126+0000 7f9ac0a59640 1 --2- 172.21.15.163:0/1693607422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9abc1472e0 0x7f9abc0b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.126+0000 7f9ac0a59640 1 --2- 172.21.15.163:0/1693607422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9abc15efb0 0x7f9abc15a3c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.126+0000 7f9ac0a59640 1 --2- 172.21.15.163:0/1693607422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa4003a50 0x7f9abc159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.126+0000 7f9ac0a59640 1 -- 172.21.15.163:0/1693607422 >> 172.21.15.163:0/1693607422 conn(0x7f9abc06c500 msgr2=0x7f9abc0737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:19.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.126+0000 7f9ac0a59640 1 -- 172.21.15.163:0/1693607422 shutdown_connections 2024-05-08T06:37:19.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.126+0000 7f9ac0a59640 1 -- 172.21.15.163:0/1693607422 wait complete. 2024-05-08T06:37:19.136 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-08T06:37:19.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.238+0000 7fe62bc20640 1 Processor -- start 2024-05-08T06:37:19.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.238+0000 7fe62bc20640 1 -- start start 2024-05-08T06:37:19.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.239+0000 7fe62bc20640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe6240d1270 0x7fe6240b0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.239+0000 7fe62bc20640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe624074370 0x7fe62406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.239+0000 7fe62bc20640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6240ae250 con 0x7fe6240d1650 2024-05-08T06:37:19.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.239+0000 7fe62bc20640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe62406f4d0 con 0x7fe6240d1270 2024-05-08T06:37:19.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.239+0000 7fe62bc20640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe6240ae000 con 0x7fe624074370 2024-05-08T06:37:19.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.239+0000 7fe62a1bd640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe6240d1650 0x7fe624072510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60652/0 (socket says 172.21.15.163:60652) 2024-05-08T06:37:19.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.239+0000 7fe62a1bd640 1 -- 172.21.15.163:0/2254641603 learned_addr learned my addr 172.21.15.163:0/2254641603 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:19.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.239+0000 7fe6299bc640 1 --2- 172.21.15.163:0/2254641603 >> v2:172.21.15.192:3300/0 conn(0x7fe6240d1270 0x7fe6240b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.239+0000 7fe6291bb640 1 --2- 172.21.15.163:0/2254641603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe624074370 0x7fe62406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.240+0000 7fe6289ba640 1 -- 172.21.15.163:0/2254641603 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3064692066 0 0) 0x7fe6240ae250 con 0x7fe6240d1650 2024-05-08T06:37:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.240+0000 7fe6289ba640 1 -- 172.21.15.163:0/2254641603 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe610003520 con 0x7fe6240d1650 2024-05-08T06:37:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.240+0000 7fe6289ba640 1 -- 172.21.15.163:0/2254641603 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2479705159 0 0) 0x7fe610003520 con 0x7fe6240d1650 2024-05-08T06:37:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.240+0000 7fe6299bc640 1 -- 172.21.15.163:0/2254641603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe624074370 msgr2=0x7fe62406ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.240+0000 7fe6299bc640 1 --2- 172.21.15.163:0/2254641603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe624074370 0x7fe62406ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.240+0000 7fe6299bc640 1 -- 172.21.15.163:0/2254641603 >> v1:172.21.15.163:6789/0 conn(0x7fe6240d1650 legacy=0x7fe624072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.240+0000 7fe6299bc640 1 -- 172.21.15.163:0/2254641603 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6240ae530 con 0x7fe6240d1270 2024-05-08T06:37:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.240+0000 7fe6291bb640 1 --2- 172.21.15.163:0/2254641603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe624074370 0x7fe62406ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.241+0000 7fe6299bc640 1 --2- 172.21.15.163:0/2254641603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6240d1270 0x7fe6240b0640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe620009d90 tx=0x7fe62002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ac74d823a468c58 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.241+0000 7fe6289ba640 1 -- 172.21.15.163:0/2254641603 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0016f0 con 0x7fe6240d1270 2024-05-08T06:37:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.241+0000 7fe6289ba640 1 -- 172.21.15.163:0/2254641603 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe620007540 con 0x7fe6240d1270 2024-05-08T06:37:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.241+0000 7fe6289ba640 1 -- 172.21.15.163:0/2254641603 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6200078f0 con 0x7fe6240d1270 2024-05-08T06:37:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.241+0000 7fe62bc20640 1 -- 172.21.15.163:0/2254641603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6240d1270 msgr2=0x7fe6240b0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.241+0000 7fe62bc20640 1 --2- 172.21.15.163:0/2254641603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6240d1270 0x7fe6240b0640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe620009d90 tx=0x7fe62002a5f0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.241+0000 7fe62bc20640 1 -- 172.21.15.163:0/2254641603 shutdown_connections 2024-05-08T06:37:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.241+0000 7fe62bc20640 1 --2- 172.21.15.163:0/2254641603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe624074370 0x7fe62406ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.241+0000 7fe62bc20640 1 --2- 172.21.15.163:0/2254641603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6240d1270 0x7fe6240b0640 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.241+0000 7fe62bc20640 1 -- 172.21.15.163:0/2254641603 >> 172.21.15.163:0/2254641603 conn(0x7fe62406c500 msgr2=0x7fe62406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.241+0000 7fe62bc20640 1 -- 172.21.15.163:0/2254641603 shutdown_connections 2024-05-08T06:37:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.241+0000 7fe62bc20640 1 -- 172.21.15.163:0/2254641603 wait complete. 2024-05-08T06:37:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.242+0000 7fe62bc20640 1 Processor -- start 2024-05-08T06:37:19.243 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.243+0000 7fe62bc20640 1 -- start start 2024-05-08T06:37:19.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.243+0000 7fe62bc20640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624074370 0x7fe6241f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.243+0000 7fe62bc20640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6240d1270 0x7fe624154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.243+0000 7fe62bc20640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6240d1650 0x7fe6240b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.243+0000 7fe62bc20640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe62406e790 con 0x7fe624074370 2024-05-08T06:37:19.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.243+0000 7fe62bc20640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe62406eb00 con 0x7fe6240d1270 2024-05-08T06:37:19.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.243+0000 7fe62bc20640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe6240d1ae0 con 0x7fe6240d1650 2024-05-08T06:37:19.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.243+0000 7fe6299bc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624074370 0x7fe6241f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.243+0000 7fe6299bc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624074370 0x7fe6241f6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40774/0 (socket says 172.21.15.163:40774) 2024-05-08T06:37:19.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.243+0000 7fe6299bc640 1 -- 172.21.15.163:0/2867984304 learned_addr learned my addr 172.21.15.163:0/2867984304 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:19.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe6291bb640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6240d1270 0x7fe624154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe62a1bd640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6240d1650 0x7fe6240b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe6299bc640 1 -- 172.21.15.163:0/2867984304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6240d1650 msgr2=0x7fe6240b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe6299bc640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6240d1650 0x7fe6240b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe6299bc640 1 -- 172.21.15.163:0/2867984304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6240d1270 msgr2=0x7fe624154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe6299bc640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6240d1270 0x7fe624154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe6299bc640 1 -- 172.21.15.163:0/2867984304 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6241f6570 con 0x7fe624074370 2024-05-08T06:37:19.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe6291bb640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6240d1270 0x7fe624154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:19.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe62a1bd640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6240d1650 0x7fe6240b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:19.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe6299bc640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624074370 0x7fe6241f6030 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe620009d90 tx=0x7fe620004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:19.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe61a7fc640 1 -- 172.21.15.163:0/2867984304 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe620040070 con 0x7fe624074370 2024-05-08T06:37:19.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe61a7fc640 1 -- 172.21.15.163:0/2867984304 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe620031c40 con 0x7fe624074370 2024-05-08T06:37:19.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe62bc20640 1 -- 172.21.15.163:0/2867984304 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6241545b0 con 0x7fe624074370 2024-05-08T06:37:19.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe61a7fc640 1 -- 172.21.15.163:0/2867984304 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe620009450 con 0x7fe624074370 2024-05-08T06:37:19.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.244+0000 7fe62bc20640 1 -- 172.21.15.163:0/2867984304 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe624152770 con 0x7fe624074370 2024-05-08T06:37:19.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.245+0000 7fe62bc20640 1 -- 172.21.15.163:0/2867984304 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe62406a550 con 0x7fe624074370 2024-05-08T06:37:19.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.246+0000 7fe61a7fc640 1 -- 172.21.15.163:0/2867984304 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe620039040 con 0x7fe624074370 2024-05-08T06:37:19.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.246+0000 7fe61a7fc640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe6080762d0 0x7fe608078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.246+0000 7fe61a7fc640 1 -- 172.21.15.163:0/2867984304 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7fe6200b82e0 con 0x7fe624074370 2024-05-08T06:37:19.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.246+0000 7fe6291bb640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe6080762d0 0x7fe608078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.247 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.247+0000 7fe6291bb640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe6080762d0 0x7fe608078790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe62409f320 tx=0x7fe614008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:19.249 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.250+0000 7fe61a7fc640 1 -- 172.21.15.163:0/2867984304 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe620081fa0 con 0x7fe624074370 2024-05-08T06:37:19.499 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.499+0000 7fe62bc20640 1 -- 172.21.15.163:0/2867984304 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fe624073b80 con 0x7fe624074370 2024-05-08T06:37:19.502 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.503+0000 7fe61a7fc640 1 -- 172.21.15.163:0/2867984304 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7fe6200818a0 con 0x7fe624074370 2024-05-08T06:37:19.505 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.505+0000 7fe62bc20640 1 -- 172.21.15.163:0/2867984304 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe6080762d0 msgr2=0x7fe608078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.505 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.505+0000 7fe62bc20640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe6080762d0 0x7fe608078790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe62409f320 tx=0x7fe614008040 comp rx=0 tx=0).stop 2024-05-08T06:37:19.505 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.505+0000 7fe62bc20640 1 -- 172.21.15.163:0/2867984304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624074370 msgr2=0x7fe6241f6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.505 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.505+0000 7fe62bc20640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624074370 0x7fe6241f6030 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe620009d90 tx=0x7fe620004a40 comp rx=0 tx=0).stop 2024-05-08T06:37:19.505 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.505+0000 7fe62bc20640 1 -- 172.21.15.163:0/2867984304 shutdown_connections 2024-05-08T06:37:19.505 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.505+0000 7fe62bc20640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe6080762d0 0x7fe608078790 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.505 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.505+0000 7fe62bc20640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6240d1650 0x7fe6240b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.505 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.505+0000 7fe62bc20640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6240d1270 0x7fe624154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.505 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.505+0000 7fe62bc20640 1 --2- 172.21.15.163:0/2867984304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624074370 0x7fe6241f6030 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.505 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.505+0000 7fe62bc20640 1 -- 172.21.15.163:0/2867984304 >> 172.21.15.163:0/2867984304 conn(0x7fe62406c500 msgr2=0x7fe6240b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:19.506 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.505+0000 7fe62bc20640 1 -- 172.21.15.163:0/2867984304 shutdown_connections 2024-05-08T06:37:19.506 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.506+0000 7fe62bc20640 1 -- 172.21.15.163:0/2867984304 wait complete. 2024-05-08T06:37:19.516 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T06:37:19.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.618+0000 7f084ce3e640 1 Processor -- start 2024-05-08T06:37:19.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.619+0000 7f084ce3e640 1 -- start start 2024-05-08T06:37:19.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.619+0000 7f084ce3e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f084815f410 0x7f08480d4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.619+0000 7f084ce3e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08480d3640 0x7f08480bab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.619+0000 7f084ce3e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08480d3b20 con 0x7f08480d0040 2024-05-08T06:37:19.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.619+0000 7f084ce3e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f08480d3e00 con 0x7f084815f410 2024-05-08T06:37:19.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.619+0000 7f084ce3e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f08480d40e0 con 0x7f08480d3640 2024-05-08T06:37:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.620+0000 7f084659c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f084815f410 0x7f08480d4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.620+0000 7f084659c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f084815f410 0x7f08480d4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53434/0 (socket says 172.21.15.163:53434) 2024-05-08T06:37:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.620+0000 7f084659c640 1 -- 172.21.15.163:0/3232126542 learned_addr learned my addr 172.21.15.163:0/3232126542 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.620+0000 7f0845d9b640 1 --2- 172.21.15.163:0/3232126542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08480d3640 0x7f08480bab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.620+0000 7f084659c640 1 -- 172.21.15.163:0/3232126542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08480d3640 msgr2=0x7f08480bab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.620+0000 7f084659c640 1 --2- 172.21.15.163:0/3232126542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08480d3640 0x7f08480bab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.620+0000 7f084659c640 1 -- 172.21.15.163:0/3232126542 >> v1:172.21.15.192:6789/0 conn(0x7f08480d0040 legacy=0x7f08480b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.620+0000 7f084659c640 1 -- 172.21.15.163:0/3232126542 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08480d43c0 con 0x7f084815f410 2024-05-08T06:37:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.621+0000 7f084659c640 1 --2- 172.21.15.163:0/3232126542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f084815f410 0x7f08480d4ba0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0840009e50 tx=0x7f084002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8308486f1db2454 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:19.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.621+0000 7f084559a640 1 -- 172.21.15.163:0/3232126542 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840036070 con 0x7f084815f410 2024-05-08T06:37:19.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.621+0000 7f084559a640 1 -- 172.21.15.163:0/3232126542 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0840031600 con 0x7f084815f410 2024-05-08T06:37:19.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.621+0000 7f084559a640 1 -- 172.21.15.163:0/3232126542 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08400319d0 con 0x7f084815f410 2024-05-08T06:37:19.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.621+0000 7f084ce3e640 1 -- 172.21.15.163:0/3232126542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f084815f410 msgr2=0x7f08480d4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.621+0000 7f084ce3e640 1 --2- 172.21.15.163:0/3232126542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f084815f410 0x7f08480d4ba0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0840009e50 tx=0x7f084002a680 comp rx=0 tx=0).stop 2024-05-08T06:37:19.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.621+0000 7f084ce3e640 1 -- 172.21.15.163:0/3232126542 shutdown_connections 2024-05-08T06:37:19.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.621+0000 7f084ce3e640 1 --2- 172.21.15.163:0/3232126542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08480d3640 0x7f08480bab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.621+0000 7f084ce3e640 1 --2- 172.21.15.163:0/3232126542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f084815f410 0x7f08480d4ba0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.621+0000 7f084ce3e640 1 -- 172.21.15.163:0/3232126542 >> 172.21.15.163:0/3232126542 conn(0x7f0848073e40 msgr2=0x7f0848074250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:19.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.621+0000 7f084ce3e640 1 -- 172.21.15.163:0/3232126542 shutdown_connections 2024-05-08T06:37:19.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.621+0000 7f084ce3e640 1 -- 172.21.15.163:0/3232126542 wait complete. 2024-05-08T06:37:19.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.623+0000 7f084ce3e640 1 Processor -- start 2024-05-08T06:37:19.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.623+0000 7f084ce3e640 1 -- start start 2024-05-08T06:37:19.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.623+0000 7f084ce3e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08480d0040 0x7f08480b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.623+0000 7f084ce3e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08480d3640 0x7f0848151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f084ce3e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f084815f410 0x7f08480c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f084ce3e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f084815f7f0 con 0x7f084815f410 2024-05-08T06:37:19.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f084ce3e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f08480bb0d0 con 0x7f08480d3640 2024-05-08T06:37:19.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f084ce3e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f084815f970 con 0x7f08480d0040 2024-05-08T06:37:19.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f0846d9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f084815f410 0x7f08480c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f0846d9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f084815f410 0x7f08480c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53438/0 (socket says 172.21.15.163:53438) 2024-05-08T06:37:19.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f0846d9d640 1 -- 172.21.15.163:0/3932837249 learned_addr learned my addr 172.21.15.163:0/3932837249 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:19.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f0845d9b640 1 --2- 172.21.15.163:0/3932837249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08480d3640 0x7f0848151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f084659c640 1 --2- 172.21.15.163:0/3932837249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08480d0040 0x7f08480b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f0846d9d640 1 -- 172.21.15.163:0/3932837249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08480d0040 msgr2=0x7f08480b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f0846d9d640 1 --2- 172.21.15.163:0/3932837249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08480d0040 0x7f08480b6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f0846d9d640 1 -- 172.21.15.163:0/3932837249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08480d3640 msgr2=0x7f0848151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f0846d9d640 1 --2- 172.21.15.163:0/3932837249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08480d3640 0x7f0848151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f0846d9d640 1 -- 172.21.15.163:0/3932837249 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08480b7460 con 0x7f084815f410 2024-05-08T06:37:19.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f0846d9d640 1 --2- 172.21.15.163:0/3932837249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f084815f410 0x7f08480c9dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f083800c440 tx=0x7f083800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:19.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f083effd640 1 -- 172.21.15.163:0/3932837249 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838011790 con 0x7f084815f410 2024-05-08T06:37:19.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f083effd640 1 -- 172.21.15.163:0/3932837249 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0838017070 con 0x7f084815f410 2024-05-08T06:37:19.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f084ce3e640 1 -- 172.21.15.163:0/3932837249 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08480b7740 con 0x7f084815f410 2024-05-08T06:37:19.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f083effd640 1 -- 172.21.15.163:0/3932837249 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838011b70 con 0x7f084815f410 2024-05-08T06:37:19.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.624+0000 7f084ce3e640 1 -- 172.21.15.163:0/3932837249 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084814c030 con 0x7f084815f410 2024-05-08T06:37:19.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.625+0000 7f084ce3e640 1 -- 172.21.15.163:0/3932837249 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08480b7c60 con 0x7f084815f410 2024-05-08T06:37:19.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.626+0000 7f083effd640 1 -- 172.21.15.163:0/3932837249 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0838011d10 con 0x7f084815f410 2024-05-08T06:37:19.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.627+0000 7f083effd640 1 --2- 172.21.15.163:0/3932837249 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f08280762d0 0x7f0828078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.627+0000 7f083effd640 1 -- 172.21.15.163:0/3932837249 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f0838096e70 con 0x7f084815f410 2024-05-08T06:37:19.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.627+0000 7f084659c640 1 --2- 172.21.15.163:0/3932837249 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f08280762d0 0x7f0828078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.627+0000 7f084659c640 1 --2- 172.21.15.163:0/3932837249 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f08280762d0 0x7f0828078790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0840033000 tx=0x7f0840009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:19.630 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.631+0000 7f083effd640 1 -- 172.21.15.163:0/3932837249 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0838060560 con 0x7f084815f410 2024-05-08T06:37:19.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.813+0000 7f084ce3e640 1 -- 172.21.15.163:0/3932837249 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f084806a4b0 con 0x7f084815f410 2024-05-08T06:37:19.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.814+0000 7f083effd640 1 -- 172.21.15.163:0/3932837249 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v27) ==== 84+0+65 (secure 0 0 0) 0x7f0838064860 con 0x7f084815f410 2024-05-08T06:37:19.813 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:19.813 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2024-05-08T06:37:19.813 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:19.813 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2024-05-08T06:37:19.813 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2024-05-08T06:37:19.813 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:19.813 INFO:teuthology.orchestra.run.smithi163.stdout:] 2024-05-08T06:37:19.815 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.816+0000 7f084ce3e640 1 -- 172.21.15.163:0/3932837249 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f08280762d0 msgr2=0x7f0828078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.815 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.816+0000 7f084ce3e640 1 --2- 172.21.15.163:0/3932837249 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f08280762d0 0x7f0828078790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0840033000 tx=0x7f0840009000 comp rx=0 tx=0).stop 2024-05-08T06:37:19.815 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.816+0000 7f084ce3e640 1 -- 172.21.15.163:0/3932837249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f084815f410 msgr2=0x7f08480c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.815 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.816+0000 7f084ce3e640 1 --2- 172.21.15.163:0/3932837249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f084815f410 0x7f08480c9dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f083800c440 tx=0x7f083800c900 comp rx=0 tx=0).stop 2024-05-08T06:37:19.815 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.816+0000 7f084ce3e640 1 -- 172.21.15.163:0/3932837249 shutdown_connections 2024-05-08T06:37:19.816 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.816+0000 7f084ce3e640 1 --2- 172.21.15.163:0/3932837249 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f08280762d0 0x7f0828078790 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.816 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.816+0000 7f084ce3e640 1 --2- 172.21.15.163:0/3932837249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f084815f410 0x7f08480c9dd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.816 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.816+0000 7f084ce3e640 1 --2- 172.21.15.163:0/3932837249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08480d3640 0x7f0848151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.816 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.816+0000 7f084ce3e640 1 --2- 172.21.15.163:0/3932837249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08480d0040 0x7f08480b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.816 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.816+0000 7f084ce3e640 1 -- 172.21.15.163:0/3932837249 >> 172.21.15.163:0/3932837249 conn(0x7f0848073e40 msgr2=0x7f08480d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:19.816 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.816+0000 7f084ce3e640 1 -- 172.21.15.163:0/3932837249 shutdown_connections 2024-05-08T06:37:19.816 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.816+0000 7f084ce3e640 1 -- 172.21.15.163:0/3932837249 wait complete. 2024-05-08T06:37:19.826 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T06:37:19.826 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T06:37:19.930 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.931+0000 7f1f2d1f2640 1 Processor -- start 2024-05-08T06:37:19.930 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.931+0000 7f1f2d1f2640 1 -- start start 2024-05-08T06:37:19.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.932+0000 7f1f2d1f2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f281578e0 0x7f1f2815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.932+0000 7f1f2d1f2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f2815e700 con 0x7f1f280b4e60 2024-05-08T06:37:19.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.932+0000 7f1f2d1f2640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f280ae160 con 0x7f1f280b52a0 2024-05-08T06:37:19.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.932+0000 7f1f2d1f2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1f280b4410 con 0x7f1f281578e0 2024-05-08T06:37:19.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.932+0000 7f1f1e59c640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1f280b4e60 0x7f1f280afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33756/0 (socket says 172.21.15.163:33756) 2024-05-08T06:37:19.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.932+0000 7f1f1e59c640 1 -- 172.21.15.163:0/1323504010 learned_addr learned my addr 172.21.15.163:0/1323504010 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:19.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.932+0000 7f1f26d9d640 1 --2- 172.21.15.163:0/1323504010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f281578e0 0x7f1f2815e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.932+0000 7f1f2659c640 1 -- 172.21.15.163:0/1323504010 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1792761748 0 0) 0x7f1f10001110 con 0x7f1f280b4e60 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2659c640 1 -- 172.21.15.163:0/1323504010 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f14003520 con 0x7f1f280b4e60 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2659c640 1 -- 172.21.15.163:0/1323504010 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4100020670 0 0) 0x7f1f14003520 con 0x7f1f280b4e60 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2659c640 1 -- 172.21.15.163:0/1323504010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f281578e0 msgr2=0x7f1f2815e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2659c640 1 --2- 172.21.15.163:0/1323504010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f281578e0 0x7f1f2815e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2659c640 1 -- 172.21.15.163:0/1323504010 >> v1:172.21.15.192:6789/0 conn(0x7f1f280b52a0 legacy=0x7f1f2806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2659c640 1 -- 172.21.15.163:0/1323504010 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f280ae420 con 0x7f1f280b4e60 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2659c640 1 -- 172.21.15.163:0/1323504010 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f1f100018c0 con 0x7f1f280b4e60 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2659c640 1 -- 172.21.15.163:0/1323504010 >> v1:172.21.15.163:6789/0 conn(0x7f1f280b4e60 legacy=0x7f1f280afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2659c640 1 --2- 172.21.15.163:0/1323504010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f281578e0 0x7f1f140039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2659c640 1 --2- 172.21.15.163:0/1323504010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f14003f70 0x7f1f14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2659c640 1 --2- 172.21.15.163:0/1323504010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f14004d40 0x7f1f140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2659c640 1 -- 172.21.15.163:0/1323504010 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1f280b4410 con 0x7f1f14004d40 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2659c640 1 -- 172.21.15.163:0/1323504010 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1400bc00 con 0x7f1f14003f70 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2659c640 1 -- 172.21.15.163:0/1323504010 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1f1400be00 con 0x7f1f281578e0 2024-05-08T06:37:19.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2759e640 1 --2- 172.21.15.163:0/1323504010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f14004d40 0x7f1f140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f26d9d640 1 --2- 172.21.15.163:0/1323504010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f281578e0 0x7f1f140039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f1e59c640 1 --2- 172.21.15.163:0/1323504010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f14003f70 0x7f1f14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2759e640 1 -- 172.21.15.163:0/1323504010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f281578e0 msgr2=0x7f1f140039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:19.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2759e640 1 --2- 172.21.15.163:0/1323504010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f281578e0 0x7f1f140039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2759e640 1 -- 172.21.15.163:0/1323504010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f14003f70 msgr2=0x7f1f14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:19.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2759e640 1 --2- 172.21.15.163:0/1323504010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f14003f70 0x7f1f14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.933+0000 7f1f2759e640 1 -- 172.21.15.163:0/1323504010 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f14003520 con 0x7f1f14004d40 2024-05-08T06:37:19.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.934+0000 7f1f2759e640 1 --2- 172.21.15.163:0/1323504010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f14004d40 0x7f1f140051c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1f20001f10 tx=0x7f1f14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52bd9318ed487f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:19.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.934+0000 7f1f2659c640 1 -- 172.21.15.163:0/1323504010 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20036070 con 0x7f1f14004d40 2024-05-08T06:37:19.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.934+0000 7f1f2659c640 1 -- 172.21.15.163:0/1323504010 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f20031600 con 0x7f1f14004d40 2024-05-08T06:37:19.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.934+0000 7f1f2659c640 1 -- 172.21.15.163:0/1323504010 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20001b30 con 0x7f1f14004d40 2024-05-08T06:37:19.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.935+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/1323504010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f14004d40 msgr2=0x7f1f140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.935+0000 7f1f2d1f2640 1 --2- 172.21.15.163:0/1323504010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f14004d40 0x7f1f140051c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1f20001f10 tx=0x7f1f14005830 comp rx=0 tx=0).stop 2024-05-08T06:37:19.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.935+0000 7f1f2759e640 1 -- 172.21.15.163:0/1323504010 reap_dead start 2024-05-08T06:37:19.937 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.935+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/1323504010 shutdown_connections 2024-05-08T06:37:19.937 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.935+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/1323504010 >> 172.21.15.163:0/1323504010 conn(0x7f1f280b2b40 msgr2=0x7f1f280b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:19.937 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.935+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/1323504010 shutdown_connections 2024-05-08T06:37:19.937 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.935+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/1323504010 wait complete. 2024-05-08T06:37:19.939 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.936+0000 7f1f2d1f2640 1 Processor -- start 2024-05-08T06:37:19.939 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.936+0000 7f1f2d1f2640 1 -- start start 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f2d1f2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f14004d40 0x7f1f28150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f2d1f2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f2814f1b0 0x7f1f2809e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f2d1f2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f280b1820 0x7f1f28144410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f2d1f2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1f280ade20 con 0x7f1f2814f1b0 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f2d1f2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1f280b4940 con 0x7f1f280b1820 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f2d1f2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1f280b6260 con 0x7f1f14004d40 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f1e59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f2814f1b0 0x7f1f2809e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f1e59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f2814f1b0 0x7f1f2809e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53462/0 (socket says 172.21.15.163:53462) 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f1e59c640 1 -- 172.21.15.163:0/3464387678 learned_addr learned my addr 172.21.15.163:0/3464387678 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f26d9d640 1 --2- 172.21.15.163:0/3464387678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f14004d40 0x7f1f28150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f2759e640 1 --2- 172.21.15.163:0/3464387678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f280b1820 0x7f1f28144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f1e59c640 1 -- 172.21.15.163:0/3464387678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f14004d40 msgr2=0x7f1f28150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f1e59c640 1 --2- 172.21.15.163:0/3464387678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f14004d40 0x7f1f28150920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f1e59c640 1 -- 172.21.15.163:0/3464387678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f280b1820 msgr2=0x7f1f28144410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:19.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f1e59c640 1 --2- 172.21.15.163:0/3464387678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f280b1820 0x7f1f28144410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:19.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.937+0000 7f1f1e59c640 1 -- 172.21.15.163:0/3464387678 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2809d4f0 con 0x7f1f2814f1b0 2024-05-08T06:37:19.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.938+0000 7f1f1e59c640 1 --2- 172.21.15.163:0/3464387678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f2814f1b0 0x7f1f2809e590 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1f10007340 tx=0x7f1f10007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:19.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.938+0000 7f1f1ffff640 1 -- 172.21.15.163:0/3464387678 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f100018a0 con 0x7f1f2814f1b0 2024-05-08T06:37:19.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.938+0000 7f1f1ffff640 1 -- 172.21.15.163:0/3464387678 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f10018070 con 0x7f1f2814f1b0 2024-05-08T06:37:19.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.938+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/3464387678 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2809d7d0 con 0x7f1f2814f1b0 2024-05-08T06:37:19.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.938+0000 7f1f1ffff640 1 -- 172.21.15.163:0/3464387678 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1000fbe0 con 0x7f1f2814f1b0 2024-05-08T06:37:19.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.938+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/3464387678 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f2809dd90 con 0x7f1f2814f1b0 2024-05-08T06:37:19.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.939+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/3464387678 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f2806a550 con 0x7f1f2814f1b0 2024-05-08T06:37:19.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.939+0000 7f1f1ffff640 1 -- 172.21.15.163:0/3464387678 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1f1001d460 con 0x7f1f2814f1b0 2024-05-08T06:37:19.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.940+0000 7f1f1ffff640 1 --2- 172.21.15.163:0/3464387678 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1f080762d0 0x7f1f08078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:19.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.940+0000 7f1f1ffff640 1 -- 172.21.15.163:0/3464387678 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f1f100971f0 con 0x7f1f2814f1b0 2024-05-08T06:37:19.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.940+0000 7f1f26d9d640 1 --2- 172.21.15.163:0/3464387678 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1f080762d0 0x7f1f08078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:19.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.940+0000 7f1f26d9d640 1 --2- 172.21.15.163:0/3464387678 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1f080762d0 0x7f1f08078790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009bc0 tx=0x7f1f18009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:19.943 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:19.944+0000 7f1f1ffff640 1 -- 172.21.15.163:0/3464387678 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1f2815e700 con 0x7f1f2814f1b0 2024-05-08T06:37:20.126 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:20.127+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/3464387678 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1f281618e0 con 0x7f1f2814f1b0 2024-05-08T06:37:20.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:20.988+0000 7f1f1ffff640 1 -- 172.21.15.163:0/3464387678 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v28) ==== 138+0+0 (secure 0 0 0) 0x7f1f10060f60 con 0x7f1f2814f1b0 2024-05-08T06:37:20.988 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2024-05-08T06:37:20.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:20.991+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/3464387678 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1f080762d0 msgr2=0x7f1f08078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:20.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:20.991+0000 7f1f2d1f2640 1 --2- 172.21.15.163:0/3464387678 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1f080762d0 0x7f1f08078790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009bc0 tx=0x7f1f18009190 comp rx=0 tx=0).stop 2024-05-08T06:37:20.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:20.991+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/3464387678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f2814f1b0 msgr2=0x7f1f2809e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:20.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:20.991+0000 7f1f2d1f2640 1 --2- 172.21.15.163:0/3464387678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f2814f1b0 0x7f1f2809e590 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1f10007340 tx=0x7f1f10007740 comp rx=0 tx=0).stop 2024-05-08T06:37:20.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:20.991+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/3464387678 shutdown_connections 2024-05-08T06:37:20.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:20.991+0000 7f1f2d1f2640 1 --2- 172.21.15.163:0/3464387678 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1f080762d0 0x7f1f08078790 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:20.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:20.991+0000 7f1f2d1f2640 1 --2- 172.21.15.163:0/3464387678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f280b1820 0x7f1f28144410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:20.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:20.991+0000 7f1f2d1f2640 1 --2- 172.21.15.163:0/3464387678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f2814f1b0 0x7f1f2809e590 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:20.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:20.991+0000 7f1f2d1f2640 1 --2- 172.21.15.163:0/3464387678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f14004d40 0x7f1f28150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:20.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:20.991+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/3464387678 >> 172.21.15.163:0/3464387678 conn(0x7f1f280b2b40 msgr2=0x7f1f2806e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:20.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:20.991+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/3464387678 shutdown_connections 2024-05-08T06:37:20.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:20.991+0000 7f1f2d1f2640 1 -- 172.21.15.163:0/3464387678 wait complete. 2024-05-08T06:37:21.008 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T06:37:21.119 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.120+0000 7fb943672640 1 Processor -- start 2024-05-08T06:37:21.120 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.121+0000 7fb943672640 1 -- start start 2024-05-08T06:37:21.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.121+0000 7fb943672640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb93c15f5a0 0x7fb93c0d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:21.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.121+0000 7fb943672640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb93c163f00 con 0x7fb93c0d19c0 2024-05-08T06:37:21.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.121+0000 7fb943672640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb93c0cfe20 con 0x7fb93c163250 2024-05-08T06:37:21.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.121+0000 7fb943672640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb93c0d0100 con 0x7fb93c15f5a0 2024-05-08T06:37:21.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.122+0000 7fb940c0d640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb93c0d19c0 0x7fb93c0d1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33760/0 (socket says 172.21.15.163:33760) 2024-05-08T06:37:21.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.122+0000 7fb940c0d640 1 -- 172.21.15.163:0/3503361605 learned_addr learned my addr 172.21.15.163:0/3503361605 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:21.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.122+0000 7fb94140e640 1 --2- 172.21.15.163:0/3503361605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb93c15f5a0 0x7fb93c0d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:21.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.122+0000 7fb93bfff640 1 -- 172.21.15.163:0/3503361605 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4064256411 0 0) 0x7fb93c163f00 con 0x7fb93c0d19c0 2024-05-08T06:37:21.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.122+0000 7fb93bfff640 1 -- 172.21.15.163:0/3503361605 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb924003520 con 0x7fb93c0d19c0 2024-05-08T06:37:21.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.122+0000 7fb93bfff640 1 -- 172.21.15.163:0/3503361605 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 264610573 0 0) 0x7fb93c0cfe20 con 0x7fb93c163250 2024-05-08T06:37:21.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.122+0000 7fb93bfff640 1 -- 172.21.15.163:0/3503361605 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb93c163f00 con 0x7fb93c163250 2024-05-08T06:37:21.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.122+0000 7fb94140e640 1 -- 172.21.15.163:0/3503361605 >> v1:172.21.15.192:6789/0 conn(0x7fb93c163250 legacy=0x7fb93c1636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:21.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.122+0000 7fb93bfff640 1 -- 172.21.15.163:0/3503361605 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 368804678 0 0) 0x7fb924003520 con 0x7fb93c0d19c0 2024-05-08T06:37:21.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.122+0000 7fb94140e640 1 -- 172.21.15.163:0/3503361605 >> v1:172.21.15.163:6789/0 conn(0x7fb93c0d19c0 legacy=0x7fb93c0d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:21.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.122+0000 7fb94140e640 1 -- 172.21.15.163:0/3503361605 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c0d03e0 con 0x7fb93c15f5a0 2024-05-08T06:37:21.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.123+0000 7fb94140e640 1 --2- 172.21.15.163:0/3503361605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb93c15f5a0 0x7fb93c0d0900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb9340097d0 tx=0x7fb93402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8c79bcf8e7209cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:21.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.123+0000 7fb93bfff640 1 -- 172.21.15.163:0/3503361605 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934037070 con 0x7fb93c15f5a0 2024-05-08T06:37:21.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.123+0000 7fb93bfff640 1 -- 172.21.15.163:0/3503361605 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb93402ac30 con 0x7fb93c15f5a0 2024-05-08T06:37:21.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.123+0000 7fb93bfff640 1 -- 172.21.15.163:0/3503361605 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9300016f0 con 0x7fb93c15f5a0 2024-05-08T06:37:21.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.123+0000 7fb943672640 1 -- 172.21.15.163:0/3503361605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb93c15f5a0 msgr2=0x7fb93c0d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:21.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.123+0000 7fb943672640 1 --2- 172.21.15.163:0/3503361605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb93c15f5a0 0x7fb93c0d0900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb9340097d0 tx=0x7fb93402a1a0 comp rx=0 tx=0).stop 2024-05-08T06:37:21.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.124+0000 7fb943672640 1 -- 172.21.15.163:0/3503361605 shutdown_connections 2024-05-08T06:37:21.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.124+0000 7fb943672640 1 --2- 172.21.15.163:0/3503361605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb93c15f5a0 0x7fb93c0d0900 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:21.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.124+0000 7fb943672640 1 -- 172.21.15.163:0/3503361605 >> 172.21.15.163:0/3503361605 conn(0x7fb93c074080 msgr2=0x7fb93c06d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:21.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.124+0000 7fb943672640 1 -- 172.21.15.163:0/3503361605 shutdown_connections 2024-05-08T06:37:21.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.124+0000 7fb943672640 1 -- 172.21.15.163:0/3503361605 wait complete. 2024-05-08T06:37:21.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.125+0000 7fb943672640 1 Processor -- start 2024-05-08T06:37:21.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.125+0000 7fb943672640 1 -- start start 2024-05-08T06:37:21.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb943672640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb93c0d19c0 0x7fb93c15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb943672640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb93c15f5a0 0x7fb93c0ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb943672640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb93c163250 0x7fb93c0d3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb943672640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c0d38e0 con 0x7fb93c163250 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb943672640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c15fb90 con 0x7fb93c0d19c0 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb943672640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb93c0d3a60 con 0x7fb93c15f5a0 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb941c0f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb93c163250 0x7fb93c0d3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb941c0f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb93c163250 0x7fb93c0d3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53464/0 (socket says 172.21.15.163:53464) 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb941c0f640 1 -- 172.21.15.163:0/2720628046 learned_addr learned my addr 172.21.15.163:0/2720628046 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb94140e640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb93c0d19c0 0x7fb93c15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb940c0d640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb93c15f5a0 0x7fb93c0ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb941c0f640 1 -- 172.21.15.163:0/2720628046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb93c15f5a0 msgr2=0x7fb93c0ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb941c0f640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb93c15f5a0 0x7fb93c0ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb941c0f640 1 -- 172.21.15.163:0/2720628046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb93c0d19c0 msgr2=0x7fb93c15ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb941c0f640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb93c0d19c0 0x7fb93c15ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb941c0f640 1 -- 172.21.15.163:0/2720628046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c1f9480 con 0x7fb93c163250 2024-05-08T06:37:21.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb940c0d640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb93c15f5a0 0x7fb93c0ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:21.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb94140e640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb93c0d19c0 0x7fb93c15ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:21.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb941c0f640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb93c163250 0x7fb93c0d3e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00c680 tx=0x7fb92c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:21.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb939ffb640 1 -- 172.21.15.163:0/2720628046 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c003b70 con 0x7fb93c163250 2024-05-08T06:37:21.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb939ffb640 1 -- 172.21.15.163:0/2720628046 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb92c003d30 con 0x7fb93c163250 2024-05-08T06:37:21.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb939ffb640 1 -- 172.21.15.163:0/2720628046 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c005310 con 0x7fb93c163250 2024-05-08T06:37:21.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.126+0000 7fb943672640 1 -- 172.21.15.163:0/2720628046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93c1f9720 con 0x7fb93c163250 2024-05-08T06:37:21.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.127+0000 7fb943672640 1 -- 172.21.15.163:0/2720628046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93c1f9ce0 con 0x7fb93c163250 2024-05-08T06:37:21.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.128+0000 7fb943672640 1 -- 172.21.15.163:0/2720628046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb93c0cf3c0 con 0x7fb93c163250 2024-05-08T06:37:21.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.128+0000 7fb939ffb640 1 -- 172.21.15.163:0/2720628046 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb92c0054b0 con 0x7fb93c163250 2024-05-08T06:37:21.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.128+0000 7fb939ffb640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb9280762d0 0x7fb928078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:21.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.129+0000 7fb939ffb640 1 -- 172.21.15.163:0/2720628046 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5227+0+0 (secure 0 0 0) 0x7fb92c097300 con 0x7fb93c163250 2024-05-08T06:37:21.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.129+0000 7fb94140e640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb9280762d0 0x7fb928078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:21.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.129+0000 7fb94140e640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb9280762d0 0x7fb928078790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb934034000 tx=0x7fb93402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:21.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.133+0000 7fb939ffb640 1 -- 172.21.15.163:0/2720628046 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb92c060e40 con 0x7fb93c163250 2024-05-08T06:37:21.313 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.314+0000 7fb943672640 1 -- 172.21.15.163:0/2720628046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb93c06a550 con 0x7fb93c163250 2024-05-08T06:37:21.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.991+0000 7fb939ffb640 1 -- 172.21.15.163:0/2720628046 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v29) ==== 170+0+0 (secure 0 0 0) 0x7fb92c060740 con 0x7fb93c163250 2024-05-08T06:37:21.990 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2024-05-08T06:37:21.993 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.994+0000 7fb943672640 1 -- 172.21.15.163:0/2720628046 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb9280762d0 msgr2=0x7fb928078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:21.993 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.994+0000 7fb943672640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb9280762d0 0x7fb928078790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb934034000 tx=0x7fb93402e000 comp rx=0 tx=0).stop 2024-05-08T06:37:21.993 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.994+0000 7fb943672640 1 -- 172.21.15.163:0/2720628046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb93c163250 msgr2=0x7fb93c0d3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:21.993 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.994+0000 7fb943672640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb93c163250 0x7fb93c0d3e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00c680 tx=0x7fb92c00cb40 comp rx=0 tx=0).stop 2024-05-08T06:37:21.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.994+0000 7fb943672640 1 -- 172.21.15.163:0/2720628046 shutdown_connections 2024-05-08T06:37:21.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.994+0000 7fb943672640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb9280762d0 0x7fb928078790 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:21.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.994+0000 7fb943672640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb93c163250 0x7fb93c0d3e50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:21.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.994+0000 7fb943672640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb93c15f5a0 0x7fb93c0ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:21.994 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.994+0000 7fb943672640 1 --2- 172.21.15.163:0/2720628046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb93c0d19c0 0x7fb93c15ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:21.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.994+0000 7fb943672640 1 -- 172.21.15.163:0/2720628046 >> 172.21.15.163:0/2720628046 conn(0x7fb93c074080 msgr2=0x7fb93c0ba670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:21.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.995+0000 7fb943672640 1 -- 172.21.15.163:0/2720628046 shutdown_connections 2024-05-08T06:37:21.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:21.995+0000 7fb943672640 1 -- 172.21.15.163:0/2720628046 wait complete. 2024-05-08T06:37:22.021 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T06:37:22.132 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.133+0000 7f1602724640 1 Processor -- start 2024-05-08T06:37:22.133 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.134+0000 7f1602724640 1 -- start start 2024-05-08T06:37:22.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.134+0000 7f1602724640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f15fc0d3720 0x7f15fc15d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:22.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.134+0000 7f1602724640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f15fc15deb0 0x7f15fc0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:22.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.134+0000 7f1602724640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15fc15e6d0 con 0x7f15fc140050 2024-05-08T06:37:22.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.134+0000 7f1602724640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f15fc071e30 con 0x7f15fc15deb0 2024-05-08T06:37:22.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.134+0000 7f1602724640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f15fc072110 con 0x7f15fc0d3720 2024-05-08T06:37:22.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.134+0000 7f1600cc1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f15fc15deb0 0x7f15fc0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:22.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.135+0000 7f15fbfff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f15fc140050 0x7f15fc0d3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33772/0 (socket says 172.21.15.163:33772) 2024-05-08T06:37:22.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.135+0000 7f1600cc1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f15fc15deb0 0x7f15fc0d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53474/0 (socket says 172.21.15.163:53474) 2024-05-08T06:37:22.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.135+0000 7f15fbfff640 1 -- 172.21.15.163:0/189427013 learned_addr learned my addr 172.21.15.163:0/189427013 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:22.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.135+0000 7f1600cc1640 1 -- 172.21.15.163:0/189427013 >> v2:172.21.15.192:3300/0 conn(0x7f15fc0d3720 msgr2=0x7f15fc15d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:22.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.135+0000 7f15fb7fe640 1 --2- 172.21.15.163:0/189427013 >> v2:172.21.15.192:3300/0 conn(0x7f15fc0d3720 0x7f15fc15d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:22.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.135+0000 7f1600cc1640 1 --2- 172.21.15.163:0/189427013 >> v2:172.21.15.192:3300/0 conn(0x7f15fc0d3720 0x7f15fc15d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:22.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.135+0000 7f15faffd640 1 -- 172.21.15.163:0/189427013 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 556239765 0 0) 0x7f15fc15e6d0 con 0x7f15fc140050 2024-05-08T06:37:22.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.135+0000 7f1600cc1640 1 -- 172.21.15.163:0/189427013 >> v1:172.21.15.163:6789/0 conn(0x7f15fc140050 legacy=0x7f15fc0d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:22.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.135+0000 7f1600cc1640 1 -- 172.21.15.163:0/189427013 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc0723f0 con 0x7f15fc15deb0 2024-05-08T06:37:22.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.136+0000 7f1600cc1640 1 --2- 172.21.15.163:0/189427013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15fc15deb0 0x7f15fc0d19f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f15fc161de0 tx=0x7f15f402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33be36d94b41ee7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:22.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.136+0000 7f15faffd640 1 -- 172.21.15.163:0/189427013 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4007740 con 0x7f15fc15deb0 2024-05-08T06:37:22.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.136+0000 7f15faffd640 1 -- 172.21.15.163:0/189427013 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f40078e0 con 0x7f15fc15deb0 2024-05-08T06:37:22.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.136+0000 7f15faffd640 1 -- 172.21.15.163:0/189427013 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4007cb0 con 0x7f15fc15deb0 2024-05-08T06:37:22.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.136+0000 7f1602724640 1 -- 172.21.15.163:0/189427013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15fc15deb0 msgr2=0x7f15fc0d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:22.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.136+0000 7f1602724640 1 --2- 172.21.15.163:0/189427013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15fc15deb0 0x7f15fc0d19f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f15fc161de0 tx=0x7f15f402a680 comp rx=0 tx=0).stop 2024-05-08T06:37:22.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.136+0000 7f1602724640 1 -- 172.21.15.163:0/189427013 shutdown_connections 2024-05-08T06:37:22.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.136+0000 7f1602724640 1 --2- 172.21.15.163:0/189427013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15fc15deb0 0x7f15fc0d19f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:22.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.136+0000 7f1602724640 1 --2- 172.21.15.163:0/189427013 >> v2:172.21.15.192:3300/0 conn(0x7f15fc0d3720 0x7f15fc15d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:22.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.136+0000 7f1602724640 1 -- 172.21.15.163:0/189427013 >> 172.21.15.163:0/189427013 conn(0x7f15fc0b4d30 msgr2=0x7f15fc0b5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:22.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.136+0000 7f1602724640 1 -- 172.21.15.163:0/189427013 shutdown_connections 2024-05-08T06:37:22.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.136+0000 7f1602724640 1 -- 172.21.15.163:0/189427013 wait complete. 2024-05-08T06:37:22.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.137+0000 7f1602724640 1 Processor -- start 2024-05-08T06:37:22.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.138+0000 7f1602724640 1 -- start start 2024-05-08T06:37:22.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.138+0000 7f1602724640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15fc0d3720 0x7f15fc143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:22.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.138+0000 7f1602724640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15fc140050 0x7f15fc0b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:22.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.138+0000 7f1602724640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15fc15deb0 0x7f15fc13c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:22.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.138+0000 7f1602724640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc072bf0 con 0x7f15fc0d3720 2024-05-08T06:37:22.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.138+0000 7f1602724640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc15e320 con 0x7f15fc15deb0 2024-05-08T06:37:22.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.138+0000 7f1602724640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f15fc144120 con 0x7f15fc140050 2024-05-08T06:37:22.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.138+0000 7f15fbfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15fc0d3720 0x7f15fc143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:22.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.138+0000 7f15fbfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15fc0d3720 0x7f15fc143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53476/0 (socket says 172.21.15.163:53476) 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.138+0000 7f15fbfff640 1 -- 172.21.15.163:0/3461385521 learned_addr learned my addr 172.21.15.163:0/3461385521 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.139+0000 7f15fbfff640 1 -- 172.21.15.163:0/3461385521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15fc140050 msgr2=0x7f15fc0b80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.139+0000 7f1600cc1640 1 --2- 172.21.15.163:0/3461385521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15fc15deb0 0x7f15fc13c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.139+0000 7f15fb7fe640 1 --2- 172.21.15.163:0/3461385521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15fc140050 0x7f15fc0b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.139+0000 7f15fbfff640 1 --2- 172.21.15.163:0/3461385521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15fc140050 0x7f15fc0b80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.139+0000 7f15fbfff640 1 -- 172.21.15.163:0/3461385521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15fc15deb0 msgr2=0x7f15fc13c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.139+0000 7f15fbfff640 1 --2- 172.21.15.163:0/3461385521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15fc15deb0 0x7f15fc13c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.139+0000 7f15fbfff640 1 -- 172.21.15.163:0/3461385521 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc144400 con 0x7f15fc0d3720 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.139+0000 7f15fbfff640 1 --2- 172.21.15.163:0/3461385521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15fc0d3720 0x7f15fc143a40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00c490 tx=0x7f15ec00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.139+0000 7f15f8ff9640 1 -- 172.21.15.163:0/3461385521 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec018070 con 0x7f15fc0d3720 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.139+0000 7f15f8ff9640 1 -- 172.21.15.163:0/3461385521 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ec007500 con 0x7f15fc0d3720 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.139+0000 7f1602724640 1 -- 172.21.15.163:0/3461385521 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15fc1446e0 con 0x7f15fc0d3720 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.139+0000 7f15f8ff9640 1 -- 172.21.15.163:0/3461385521 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec004a20 con 0x7f15fc0d3720 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.139+0000 7f1602724640 1 -- 172.21.15.163:0/3461385521 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15fc144ca0 con 0x7f15fc0d3720 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.140+0000 7f1602724640 1 -- 172.21.15.163:0/3461385521 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15fc06a550 con 0x7f15fc0d3720 2024-05-08T06:37:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.141+0000 7f15f8ff9640 1 -- 172.21.15.163:0/3461385521 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f15ec01b070 con 0x7f15fc0d3720 2024-05-08T06:37:22.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.141+0000 7f15f8ff9640 1 --2- 172.21.15.163:0/3461385521 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f15e80762d0 0x7f15e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:22.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.141+0000 7f15fb7fe640 1 --2- 172.21.15.163:0/3461385521 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f15e80762d0 0x7f15e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:22.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.141+0000 7f15f8ff9640 1 -- 172.21.15.163:0/3461385521 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5609+0+0 (secure 0 0 0) 0x7f15ec0968d0 con 0x7f15fc0d3720 2024-05-08T06:37:22.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.142+0000 7f15fb7fe640 1 --2- 172.21.15.163:0/3461385521 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f15e80762d0 0x7f15e8078790 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f15f000b7c0 tx=0x7f15f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:22.145 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.146+0000 7f15f8ff9640 1 -- 172.21.15.163:0/3461385521 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f15ec060290 con 0x7f15fc0d3720 2024-05-08T06:37:22.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:22.364+0000 7f1602724640 1 -- 172.21.15.163:0/3461385521 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f15fc0d3d60 con 0x7f15fc0d3720 2024-05-08T06:37:23.017 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 4 and data pool 5 2024-05-08T06:37:23.017 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.016+0000 7f15f8ff9640 1 -- 172.21.15.163:0/3461385521 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7f15ec05fb90 con 0x7f15fc0d3720 2024-05-08T06:37:23.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.021+0000 7f1602724640 1 -- 172.21.15.163:0/3461385521 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f15e80762d0 msgr2=0x7f15e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.021+0000 7f1602724640 1 --2- 172.21.15.163:0/3461385521 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f15e80762d0 0x7f15e8078790 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f15f000b7c0 tx=0x7f15f0008040 comp rx=0 tx=0).stop 2024-05-08T06:37:23.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.021+0000 7f1602724640 1 -- 172.21.15.163:0/3461385521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15fc0d3720 msgr2=0x7f15fc143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.021+0000 7f1602724640 1 --2- 172.21.15.163:0/3461385521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15fc0d3720 0x7f15fc143a40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00c490 tx=0x7f15ec00c950 comp rx=0 tx=0).stop 2024-05-08T06:37:23.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.022+0000 7f1602724640 1 -- 172.21.15.163:0/3461385521 shutdown_connections 2024-05-08T06:37:23.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.022+0000 7f1602724640 1 --2- 172.21.15.163:0/3461385521 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f15e80762d0 0x7f15e8078790 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.022+0000 7f1602724640 1 --2- 172.21.15.163:0/3461385521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15fc15deb0 0x7f15fc13c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.022+0000 7f1602724640 1 --2- 172.21.15.163:0/3461385521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15fc140050 0x7f15fc0b80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.022+0000 7f1602724640 1 --2- 172.21.15.163:0/3461385521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15fc0d3720 0x7f15fc143a40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.022+0000 7f1602724640 1 -- 172.21.15.163:0/3461385521 >> 172.21.15.163:0/3461385521 conn(0x7f15fc0b4d30 msgr2=0x7f15fc15fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:23.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.022+0000 7f1602724640 1 -- 172.21.15.163:0/3461385521 shutdown_connections 2024-05-08T06:37:23.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.023+0000 7f1602724640 1 -- 172.21.15.163:0/3461385521 wait complete. 2024-05-08T06:37:23.047 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:37:23.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.149+0000 7f8c86ae1640 1 Processor -- start 2024-05-08T06:37:23.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.150+0000 7f8c86ae1640 1 -- start start 2024-05-08T06:37:23.150 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.150+0000 7f8c86ae1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8c800d0a00 0x7f8c800b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.150 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.150+0000 7f8c86ae1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c800d0080 con 0x7f8c800cfc10 2024-05-08T06:37:23.150 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.150+0000 7f8c86ae1640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c800d36f0 con 0x7f8c8015f760 2024-05-08T06:37:23.150 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.150+0000 7f8c86ae1640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f8c8015fb40 con 0x7f8c800d0a00 2024-05-08T06:37:23.150 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.151+0000 7f8c7ffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8c800d0a00 0x7f8c800b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.151 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.151+0000 7f8c8507e640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8c800cfc10 0x7f8c800b8710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33780/0 (socket says 172.21.15.163:33780) 2024-05-08T06:37:23.151 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.151+0000 7f8c8507e640 1 -- 172.21.15.163:0/1156497939 learned_addr learned my addr 172.21.15.163:0/1156497939 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:23.151 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.151+0000 7f8c7ffff640 1 -- 172.21.15.163:0/1156497939 >> v1:172.21.15.192:6789/0 conn(0x7f8c8015f760 legacy=0x7f8c800d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:23.151 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.151+0000 7f8c7ffff640 1 -- 172.21.15.163:0/1156497939 >> v1:172.21.15.163:6789/0 conn(0x7f8c800cfc10 legacy=0x7f8c800b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.151 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.151+0000 7f8c7ffff640 1 -- 172.21.15.163:0/1156497939 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c800d39d0 con 0x7f8c800d0a00 2024-05-08T06:37:23.151 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.151+0000 7f8c7f7fe640 1 -- 172.21.15.163:0/1156497939 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3005619513 0 0) 0x7f8c800d0080 con 0x7f8c800cfc10 2024-05-08T06:37:23.151 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.152+0000 7f8c7ffff640 1 --2- 172.21.15.163:0/1156497939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c800d0a00 0x7f8c800b4770 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0097a0 tx=0x7f8c6c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d60b4e97adc87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:23.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.152+0000 7f8c7f7fe640 1 -- 172.21.15.163:0/1156497939 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c037070 con 0x7f8c800d0a00 2024-05-08T06:37:23.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.152+0000 7f8c7f7fe640 1 -- 172.21.15.163:0/1156497939 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c02ae50 con 0x7f8c800d0a00 2024-05-08T06:37:23.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.152+0000 7f8c7f7fe640 1 -- 172.21.15.163:0/1156497939 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c032660 con 0x7f8c800d0a00 2024-05-08T06:37:23.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.152+0000 7f8c86ae1640 1 -- 172.21.15.163:0/1156497939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c800d0a00 msgr2=0x7f8c800b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.152+0000 7f8c86ae1640 1 --2- 172.21.15.163:0/1156497939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c800d0a00 0x7f8c800b4770 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0097a0 tx=0x7f8c6c02a100 comp rx=0 tx=0).stop 2024-05-08T06:37:23.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.152+0000 7f8c86ae1640 1 -- 172.21.15.163:0/1156497939 shutdown_connections 2024-05-08T06:37:23.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.152+0000 7f8c86ae1640 1 --2- 172.21.15.163:0/1156497939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c800d0a00 0x7f8c800b4770 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.152+0000 7f8c86ae1640 1 -- 172.21.15.163:0/1156497939 >> 172.21.15.163:0/1156497939 conn(0x7f8c800740e0 msgr2=0x7f8c8006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:23.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.152+0000 7f8c86ae1640 1 -- 172.21.15.163:0/1156497939 shutdown_connections 2024-05-08T06:37:23.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.152+0000 7f8c86ae1640 1 -- 172.21.15.163:0/1156497939 wait complete. 2024-05-08T06:37:23.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.153+0000 7f8c86ae1640 1 Processor -- start 2024-05-08T06:37:23.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.153+0000 7f8c86ae1640 1 -- start start 2024-05-08T06:37:23.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.154+0000 7f8c86ae1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c800cfc10 0x7f8c8014ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.154+0000 7f8c86ae1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c800d0a00 0x7f8c801e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.154+0000 7f8c86ae1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8015f760 0x7f8c800ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.154+0000 7f8c86ae1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8c800d4270 con 0x7f8c8015f760 2024-05-08T06:37:23.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.154+0000 7f8c86ae1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c800d3f90 con 0x7f8c800cfc10 2024-05-08T06:37:23.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.154+0000 7f8c86ae1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8c800d4790 con 0x7f8c800d0a00 2024-05-08T06:37:23.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c8507e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8015f760 0x7f8c800ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c8507e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8015f760 0x7f8c800ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53486/0 (socket says 172.21.15.163:53486) 2024-05-08T06:37:23.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c8507e640 1 -- 172.21.15.163:0/2379247200 learned_addr learned my addr 172.21.15.163:0/2379247200 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:23.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c8487d640 1 --2- 172.21.15.163:0/2379247200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c800cfc10 0x7f8c8014ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c7ffff640 1 --2- 172.21.15.163:0/2379247200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c800d0a00 0x7f8c801e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c8507e640 1 -- 172.21.15.163:0/2379247200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c800d0a00 msgr2=0x7f8c801e71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c8507e640 1 --2- 172.21.15.163:0/2379247200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c800d0a00 0x7f8c801e71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c8507e640 1 -- 172.21.15.163:0/2379247200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c800cfc10 msgr2=0x7f8c8014ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c8507e640 1 --2- 172.21.15.163:0/2379247200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c800cfc10 0x7f8c8014ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c8507e640 1 -- 172.21.15.163:0/2379247200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c801f8920 con 0x7f8c8015f760 2024-05-08T06:37:23.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c8507e640 1 --2- 172.21.15.163:0/2379247200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8015f760 0x7f8c800ce850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800c6b0 tx=0x7f8c7800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:23.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c7d7fa640 1 -- 172.21.15.163:0/2379247200 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c78018070 con 0x7f8c8015f760 2024-05-08T06:37:23.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c7d7fa640 1 -- 172.21.15.163:0/2379247200 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c78004da0 con 0x7f8c8015f760 2024-05-08T06:37:23.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c7d7fa640 1 -- 172.21.15.163:0/2379247200 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800a880 con 0x7f8c8015f760 2024-05-08T06:37:23.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c86ae1640 1 -- 172.21.15.163:0/2379247200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c801e7c30 con 0x7f8c8015f760 2024-05-08T06:37:23.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.155+0000 7f8c86ae1640 1 -- 172.21.15.163:0/2379247200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c801f8ec0 con 0x7f8c8015f760 2024-05-08T06:37:23.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.156+0000 7f8c86ae1640 1 -- 172.21.15.163:0/2379247200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c8006a550 con 0x7f8c8015f760 2024-05-08T06:37:23.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.157+0000 7f8c7d7fa640 1 -- 172.21.15.163:0/2379247200 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8c7801b070 con 0x7f8c8015f760 2024-05-08T06:37:23.158 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.157+0000 7f8c7d7fa640 1 --2- 172.21.15.163:0/2379247200 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8c640762d0 0x7f8c64078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.158 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.157+0000 7f8c7d7fa640 1 -- 172.21.15.163:0/2379247200 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f8c78096ad0 con 0x7f8c8015f760 2024-05-08T06:37:23.158 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.157+0000 7f8c8487d640 1 --2- 172.21.15.163:0/2379247200 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8c640762d0 0x7f8c64078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.159 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.158+0000 7f8c8487d640 1 --2- 172.21.15.163:0/2379247200 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8c640762d0 0x7f8c64078790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8c74004f60 tx=0x7f8c74007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:23.160 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.162+0000 7f8c7d7fa640 1 -- 172.21.15.163:0/2379247200 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8c7805fe70 con 0x7f8c8015f760 2024-05-08T06:37:23.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.341+0000 7f8c86ae1640 1 -- 172.21.15.163:0/2379247200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c800cc570 con 0x7f8c8015f760 2024-05-08T06:37:23.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.342+0000 7f8c7d7fa640 1 -- 172.21.15.163:0/2379247200 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16706 (secure 0 0 0) 0x7f8c78064170 con 0x7f8c8015f760 2024-05-08T06:37:23.342 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:23.342 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":30,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:37:22.365490+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:37:20.131664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T06:37:21.318802+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:37:23.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.344+0000 7f8c86ae1640 1 -- 172.21.15.163:0/2379247200 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8c640762d0 msgr2=0x7f8c64078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.344+0000 7f8c86ae1640 1 --2- 172.21.15.163:0/2379247200 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8c640762d0 0x7f8c64078790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8c74004f60 tx=0x7f8c74007e30 comp rx=0 tx=0).stop 2024-05-08T06:37:23.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.344+0000 7f8c86ae1640 1 -- 172.21.15.163:0/2379247200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8015f760 msgr2=0x7f8c800ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.344+0000 7f8c86ae1640 1 --2- 172.21.15.163:0/2379247200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8015f760 0x7f8c800ce850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800c6b0 tx=0x7f8c7800cb70 comp rx=0 tx=0).stop 2024-05-08T06:37:23.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.344+0000 7f8c86ae1640 1 -- 172.21.15.163:0/2379247200 shutdown_connections 2024-05-08T06:37:23.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.344+0000 7f8c86ae1640 1 --2- 172.21.15.163:0/2379247200 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8c640762d0 0x7f8c64078790 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.344+0000 7f8c86ae1640 1 --2- 172.21.15.163:0/2379247200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8015f760 0x7f8c800ce850 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.344+0000 7f8c86ae1640 1 --2- 172.21.15.163:0/2379247200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c800d0a00 0x7f8c801e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.344+0000 7f8c86ae1640 1 --2- 172.21.15.163:0/2379247200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c800cfc10 0x7f8c8014ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.344+0000 7f8c86ae1640 1 -- 172.21.15.163:0/2379247200 >> 172.21.15.163:0/2379247200 conn(0x7f8c800740e0 msgr2=0x7f8c800ba6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:23.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.344+0000 7f8c86ae1640 1 -- 172.21.15.163:0/2379247200 shutdown_connections 2024-05-08T06:37:23.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.344+0000 7f8c86ae1640 1 -- 172.21.15.163:0/2379247200 wait complete. 2024-05-08T06:37:23.355 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:37:23.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.456+0000 7f0da559d640 1 Processor -- start 2024-05-08T06:37:23.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.456+0000 7f0da559d640 1 -- start start 2024-05-08T06:37:23.457 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.457+0000 7f0da559d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0da0163bd0 0x7f0da00d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.457 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.457+0000 7f0da559d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0da00d0ed0 0x7f0da00d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.457 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.457+0000 7f0da559d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da00d6040 con 0x7f0da015f3a0 2024-05-08T06:37:23.457 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.457+0000 7f0da559d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0da00d0a40 con 0x7f0da00d0ed0 2024-05-08T06:37:23.457 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.457+0000 7f0da559d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0da015f780 con 0x7f0da0163bd0 2024-05-08T06:37:23.457 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.457+0000 7f0d9effd640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0da015f3a0 0x7f0da00d5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33796/0 (socket says 172.21.15.163:33796) 2024-05-08T06:37:23.457 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.457+0000 7f0d9effd640 1 -- 172.21.15.163:0/764496867 learned_addr learned my addr 172.21.15.163:0/764496867 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:23.457 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.457+0000 7f0d9f7fe640 1 --2- 172.21.15.163:0/764496867 >> v2:172.21.15.192:3300/0 conn(0x7f0da00d0ed0 0x7f0da00d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.457 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.457+0000 7f0d9e7fc640 1 --2- 172.21.15.163:0/764496867 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0da0163bd0 0x7f0da00d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.458 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.458+0000 7f0d9dffb640 1 -- 172.21.15.163:0/764496867 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1150752842 0 0) 0x7f0da00d6040 con 0x7f0da015f3a0 2024-05-08T06:37:23.458 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.458+0000 7f0d9dffb640 1 -- 172.21.15.163:0/764496867 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d88003520 con 0x7f0da015f3a0 2024-05-08T06:37:23.458 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.458+0000 7f0d9f7fe640 1 -- 172.21.15.163:0/764496867 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0da0163bd0 msgr2=0x7f0da00d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:23.458 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.458+0000 7f0d9f7fe640 1 --2- 172.21.15.163:0/764496867 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0da0163bd0 0x7f0da00d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.458 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.458+0000 7f0d9f7fe640 1 -- 172.21.15.163:0/764496867 >> v1:172.21.15.163:6789/0 conn(0x7f0da015f3a0 legacy=0x7f0da00d5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.458 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.458+0000 7f0d9dffb640 1 -- 172.21.15.163:0/764496867 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3695917521 0 0) 0x7f0d88003520 con 0x7f0da015f3a0 2024-05-08T06:37:23.458 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.458+0000 7f0d9f7fe640 1 -- 172.21.15.163:0/764496867 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da00d0110 con 0x7f0da00d0ed0 2024-05-08T06:37:23.458 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.458+0000 7f0d9e7fc640 1 --2- 172.21.15.163:0/764496867 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0da0163bd0 0x7f0da00d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:23.460 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.459+0000 7f0d9f7fe640 1 --2- 172.21.15.163:0/764496867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00d0ed0 0x7f0da00d2170 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0da00d0bc0 tx=0x7f0d9802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=888bc657fee8b50b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:23.460 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.459+0000 7f0d9dffb640 1 -- 172.21.15.163:0/764496867 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98037070 con 0x7f0da00d0ed0 2024-05-08T06:37:23.460 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.459+0000 7f0d9dffb640 1 -- 172.21.15.163:0/764496867 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9802aba0 con 0x7f0da00d0ed0 2024-05-08T06:37:23.460 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.459+0000 7f0d9dffb640 1 -- 172.21.15.163:0/764496867 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90001a70 con 0x7f0da00d0ed0 2024-05-08T06:37:23.460 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.459+0000 7f0da559d640 1 -- 172.21.15.163:0/764496867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00d0ed0 msgr2=0x7f0da00d2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.460 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.459+0000 7f0da559d640 1 --2- 172.21.15.163:0/764496867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00d0ed0 0x7f0da00d2170 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0da00d0bc0 tx=0x7f0d9802a110 comp rx=0 tx=0).stop 2024-05-08T06:37:23.460 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.459+0000 7f0da559d640 1 -- 172.21.15.163:0/764496867 shutdown_connections 2024-05-08T06:37:23.460 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.459+0000 7f0da559d640 1 --2- 172.21.15.163:0/764496867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00d0ed0 0x7f0da00d2170 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.460 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.459+0000 7f0da559d640 1 --2- 172.21.15.163:0/764496867 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0da0163bd0 0x7f0da00d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.460 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.459+0000 7f0da559d640 1 -- 172.21.15.163:0/764496867 >> 172.21.15.163:0/764496867 conn(0x7f0da009e5b0 msgr2=0x7f0da00d5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:23.460 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.459+0000 7f0da559d640 1 -- 172.21.15.163:0/764496867 shutdown_connections 2024-05-08T06:37:23.460 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.459+0000 7f0da559d640 1 -- 172.21.15.163:0/764496867 wait complete. 2024-05-08T06:37:23.460 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.461+0000 7f0da559d640 1 Processor -- start 2024-05-08T06:37:23.460 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.461+0000 7f0da559d640 1 -- start start 2024-05-08T06:37:23.462 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.461+0000 7f0da559d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00d0ed0 0x7f0da00ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.461+0000 7f0da559d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da015f3a0 0x7f0da0158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.461+0000 7f0da559d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0da0163bd0 0x7f0da00b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.461+0000 7f0da559d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0da00cfdc0 con 0x7f0da015f3a0 2024-05-08T06:37:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.461+0000 7f0da559d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0da00d06d0 con 0x7f0da00d0ed0 2024-05-08T06:37:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.461+0000 7f0da559d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0da006d3e0 con 0x7f0da0163bd0 2024-05-08T06:37:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d9e7fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da015f3a0 0x7f0da0158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d9e7fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da015f3a0 0x7f0da0158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53500/0 (socket says 172.21.15.163:53500) 2024-05-08T06:37:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d9e7fc640 1 -- 172.21.15.163:0/3814605454 learned_addr learned my addr 172.21.15.163:0/3814605454 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d9effd640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00d0ed0 0x7f0da00ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d9f7fe640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0da0163bd0 0x7f0da00b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d9e7fc640 1 -- 172.21.15.163:0/3814605454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0da0163bd0 msgr2=0x7f0da00b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d9e7fc640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0da0163bd0 0x7f0da00b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d9e7fc640 1 -- 172.21.15.163:0/3814605454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00d0ed0 msgr2=0x7f0da00ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d9e7fc640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00d0ed0 0x7f0da00ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.464 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d9e7fc640 1 -- 172.21.15.163:0/3814605454 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da00ceda0 con 0x7f0da015f3a0 2024-05-08T06:37:23.464 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d9effd640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00d0ed0 0x7f0da00ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:23.464 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d9e7fc640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da015f3a0 0x7f0da0158980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400e990 tx=0x7f0d9400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:23.464 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d877fe640 1 -- 172.21.15.163:0/3814605454 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94017070 con 0x7f0da015f3a0 2024-05-08T06:37:23.464 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d877fe640 1 -- 172.21.15.163:0/3814605454 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9400c9e0 con 0x7f0da015f3a0 2024-05-08T06:37:23.464 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0da559d640 1 -- 172.21.15.163:0/3814605454 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da00cf040 con 0x7f0da015f3a0 2024-05-08T06:37:23.464 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0d877fe640 1 -- 172.21.15.163:0/3814605454 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9400cdb0 con 0x7f0da015f3a0 2024-05-08T06:37:23.464 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.462+0000 7f0da559d640 1 -- 172.21.15.163:0/3814605454 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da00b8340 con 0x7f0da015f3a0 2024-05-08T06:37:23.464 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.463+0000 7f0da559d640 1 -- 172.21.15.163:0/3814605454 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da00b8a60 con 0x7f0da015f3a0 2024-05-08T06:37:23.465 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.464+0000 7f0d877fe640 1 -- 172.21.15.163:0/3814605454 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0d9401a070 con 0x7f0da015f3a0 2024-05-08T06:37:23.465 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.464+0000 7f0d877fe640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0d8c0762d0 0x7f0d8c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.465 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.465+0000 7f0d877fe640 1 -- 172.21.15.163:0/3814605454 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f0d94097510 con 0x7f0da015f3a0 2024-05-08T06:37:23.465 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.465+0000 7f0d9effd640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0d8c0762d0 0x7f0d8c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.465 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.465+0000 7f0d9effd640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0d8c0762d0 0x7f0d8c078790 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0d90002230 tx=0x7f0d90009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:23.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.469+0000 7f0d877fe640 1 -- 172.21.15.163:0/3814605454 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0d94060de0 con 0x7f0da015f3a0 2024-05-08T06:37:23.652 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.653+0000 7f0da559d640 1 -- 172.21.15.163:0/3814605454 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0da006a550 con 0x7f0da015f3a0 2024-05-08T06:37:23.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.654+0000 7f0d877fe640 1 -- 172.21.15.163:0/3814605454 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16706 (secure 0 0 0) 0x7f0d940606e0 con 0x7f0da015f3a0 2024-05-08T06:37:23.654 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:23.654 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":30,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:37:22.365490+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:37:20.131664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T06:37:21.318802+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:37:23.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.656+0000 7f0da559d640 1 -- 172.21.15.163:0/3814605454 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0d8c0762d0 msgr2=0x7f0d8c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.656+0000 7f0da559d640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0d8c0762d0 0x7f0d8c078790 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0d90002230 tx=0x7f0d90009040 comp rx=0 tx=0).stop 2024-05-08T06:37:23.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.656+0000 7f0da559d640 1 -- 172.21.15.163:0/3814605454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da015f3a0 msgr2=0x7f0da0158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.656+0000 7f0da559d640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da015f3a0 0x7f0da0158980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400e990 tx=0x7f0d9400ee50 comp rx=0 tx=0).stop 2024-05-08T06:37:23.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.656+0000 7f0da559d640 1 -- 172.21.15.163:0/3814605454 shutdown_connections 2024-05-08T06:37:23.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.657+0000 7f0da559d640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0d8c0762d0 0x7f0d8c078790 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.657+0000 7f0da559d640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0da0163bd0 0x7f0da00b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.657+0000 7f0da559d640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0da015f3a0 0x7f0da0158980 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.657+0000 7f0da559d640 1 --2- 172.21.15.163:0/3814605454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0da00d0ed0 0x7f0da00ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.657+0000 7f0da559d640 1 -- 172.21.15.163:0/3814605454 >> 172.21.15.163:0/3814605454 conn(0x7f0da009e5b0 msgr2=0x7f0da015f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:23.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.657+0000 7f0da559d640 1 -- 172.21.15.163:0/3814605454 shutdown_connections 2024-05-08T06:37:23.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.657+0000 7f0da559d640 1 -- 172.21.15.163:0/3814605454 wait complete. 2024-05-08T06:37:23.667 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T06:37:23.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.771+0000 7fc0f2bc1640 1 Processor -- start 2024-05-08T06:37:23.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.771+0000 7fc0f2bc1640 1 -- start start 2024-05-08T06:37:23.772 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.771+0000 7fc0f2bc1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec140100 0x7fc0ec070670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.772 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.771+0000 7fc0f2bc1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc0ec161510 0x7fc0ec0d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.772 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.771+0000 7fc0f2bc1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0ec15d7a0 con 0x7fc0ec070bb0 2024-05-08T06:37:23.772 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.771+0000 7fc0f2bc1640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fc0ec161b70 con 0x7fc0ec161510 2024-05-08T06:37:23.772 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.771+0000 7fc0f2bc1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ec15da60 con 0x7fc0ec140100 2024-05-08T06:37:23.772 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.772+0000 7fc0f115e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc0ec161510 0x7fc0ec0d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.772 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.772+0000 7fc0ebfff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc0ec070bb0 0x7fc0ec160e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33798/0 (socket says 172.21.15.163:33798) 2024-05-08T06:37:23.772 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.772+0000 7fc0ebfff640 1 -- 172.21.15.163:0/2380444493 learned_addr learned my addr 172.21.15.163:0/2380444493 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:23.772 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.772+0000 7fc0f095d640 1 --2- 172.21.15.163:0/2380444493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec140100 0x7fc0ec070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.772 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.772+0000 7fc0f115e640 1 -- 172.21.15.163:0/2380444493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec140100 msgr2=0x7fc0ec070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:23.772 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.772+0000 7fc0f115e640 1 --2- 172.21.15.163:0/2380444493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec140100 0x7fc0ec070670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.772+0000 7fc0f115e640 1 -- 172.21.15.163:0/2380444493 >> v1:172.21.15.163:6789/0 conn(0x7fc0ec070bb0 legacy=0x7fc0ec160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.772+0000 7fc0eb7fe640 1 -- 172.21.15.163:0/2380444493 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3420309851 0 0) 0x7fc0ec15d7a0 con 0x7fc0ec070bb0 2024-05-08T06:37:23.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.772+0000 7fc0f115e640 1 -- 172.21.15.163:0/2380444493 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec15dd40 con 0x7fc0ec161510 2024-05-08T06:37:23.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.773+0000 7fc0f115e640 1 --2- 172.21.15.163:0/2380444493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec161510 0x7fc0ec0d19c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009d90 tx=0x7fc0e402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b88af0ac7164c9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:23.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.773+0000 7fc0eb7fe640 1 -- 172.21.15.163:0/2380444493 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4031600 con 0x7fc0ec161510 2024-05-08T06:37:23.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.773+0000 7fc0eb7fe640 1 -- 172.21.15.163:0/2380444493 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e40317a0 con 0x7fc0ec161510 2024-05-08T06:37:23.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.773+0000 7fc0eb7fe640 1 -- 172.21.15.163:0/2380444493 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4031b70 con 0x7fc0ec161510 2024-05-08T06:37:23.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.773+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/2380444493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec161510 msgr2=0x7fc0ec0d19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.773+0000 7fc0f2bc1640 1 --2- 172.21.15.163:0/2380444493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec161510 0x7fc0ec0d19c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009d90 tx=0x7fc0e402a5d0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.773+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/2380444493 shutdown_connections 2024-05-08T06:37:23.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.773+0000 7fc0f2bc1640 1 --2- 172.21.15.163:0/2380444493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec161510 0x7fc0ec0d19c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.773+0000 7fc0f2bc1640 1 --2- 172.21.15.163:0/2380444493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec140100 0x7fc0ec070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.773+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/2380444493 >> 172.21.15.163:0/2380444493 conn(0x7fc0ec072a60 msgr2=0x7fc0ec06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:23.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.773+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/2380444493 shutdown_connections 2024-05-08T06:37:23.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.773+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/2380444493 wait complete. 2024-05-08T06:37:23.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.775+0000 7fc0f2bc1640 1 Processor -- start 2024-05-08T06:37:23.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.775+0000 7fc0f2bc1640 1 -- start start 2024-05-08T06:37:23.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.775+0000 7fc0f2bc1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec070bb0 0x7fc0ec142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.775+0000 7fc0f2bc1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec140100 0x7fc0ec1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.775+0000 7fc0f2bc1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec161510 0x7fc0ec139910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.775+0000 7fc0f2bc1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec15e540 con 0x7fc0ec161510 2024-05-08T06:37:23.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.775+0000 7fc0f2bc1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec1404e0 con 0x7fc0ec140100 2024-05-08T06:37:23.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.775+0000 7fc0f2bc1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ec15e6c0 con 0x7fc0ec070bb0 2024-05-08T06:37:23.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0f115e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec161510 0x7fc0ec139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0f115e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec161510 0x7fc0ec139910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53522/0 (socket says 172.21.15.163:53522) 2024-05-08T06:37:23.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0f115e640 1 -- 172.21.15.163:0/3841846055 learned_addr learned my addr 172.21.15.163:0/3841846055 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:23.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0f095d640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec070bb0 0x7fc0ec142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0ebfff640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec140100 0x7fc0ec1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0f115e640 1 -- 172.21.15.163:0/3841846055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec070bb0 msgr2=0x7fc0ec142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0f115e640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec070bb0 0x7fc0ec142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0f115e640 1 -- 172.21.15.163:0/3841846055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec140100 msgr2=0x7fc0ec1445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:23.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0f115e640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec140100 0x7fc0ec1445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:23.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0f115e640 1 -- 172.21.15.163:0/3841846055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec0b5380 con 0x7fc0ec161510 2024-05-08T06:37:23.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0ebfff640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec140100 0x7fc0ec1445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:23.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0f115e640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec161510 0x7fc0ec139910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009eb0 tx=0x7fc0e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:23.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0e97fa640 1 -- 172.21.15.163:0/3841846055 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4040070 con 0x7fc0ec161510 2024-05-08T06:37:23.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0e97fa640 1 -- 172.21.15.163:0/3841846055 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e4030be0 con 0x7fc0ec161510 2024-05-08T06:37:23.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/3841846055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ec13d850 con 0x7fc0ec161510 2024-05-08T06:37:23.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0e97fa640 1 -- 172.21.15.163:0/3841846055 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4009450 con 0x7fc0ec161510 2024-05-08T06:37:23.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.776+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/3841846055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ec0b5900 con 0x7fc0ec161510 2024-05-08T06:37:23.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.778+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/3841846055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ec06a550 con 0x7fc0ec161510 2024-05-08T06:37:23.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.778+0000 7fc0e97fa640 1 -- 172.21.15.163:0/3841846055 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc0e4039040 con 0x7fc0ec161510 2024-05-08T06:37:23.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.779+0000 7fc0e97fa640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc0d00762d0 0x7fc0d0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:23.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.779+0000 7fc0f095d640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc0d00762d0 0x7fc0d0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:23.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.779+0000 7fc0e97fa640 1 -- 172.21.15.163:0/3841846055 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fc0e40b8290 con 0x7fc0ec161510 2024-05-08T06:37:23.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.779+0000 7fc0f095d640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc0d00762d0 0x7fc0d0078790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0006fd0 tx=0x7fc0e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:23.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.783+0000 7fc0e97fa640 1 -- 172.21.15.163:0/3841846055 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc0e4081c80 con 0x7fc0ec161510 2024-05-08T06:37:23.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:23.998+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/3841846055 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc0ec13ef20 con 0x7fc0ec161510 2024-05-08T06:37:24.017 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.018+0000 7fc0e97fa640 1 -- 172.21.15.163:0/3841846055 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) ==== 118+0+0 (secure 0 0 0) 0x7fc0e4081580 con 0x7fc0ec161510 2024-05-08T06:37:24.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.020+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/3841846055 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc0d00762d0 msgr2=0x7fc0d0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:24.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.020+0000 7fc0f2bc1640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc0d00762d0 0x7fc0d0078790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0006fd0 tx=0x7fc0e0008040 comp rx=0 tx=0).stop 2024-05-08T06:37:24.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.020+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/3841846055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec161510 msgr2=0x7fc0ec139910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:24.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.020+0000 7fc0f2bc1640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec161510 0x7fc0ec139910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009eb0 tx=0x7fc0e4004a40 comp rx=0 tx=0).stop 2024-05-08T06:37:24.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.020+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/3841846055 shutdown_connections 2024-05-08T06:37:24.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.020+0000 7fc0f2bc1640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc0d00762d0 0x7fc0d0078790 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:24.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.020+0000 7fc0f2bc1640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec161510 0x7fc0ec139910 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:24.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.020+0000 7fc0f2bc1640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec140100 0x7fc0ec1445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:24.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.020+0000 7fc0f2bc1640 1 --2- 172.21.15.163:0/3841846055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec070bb0 0x7fc0ec142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:24.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.020+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/3841846055 >> 172.21.15.163:0/3841846055 conn(0x7fc0ec072a60 msgr2=0x7fc0ec15f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:24.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.020+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/3841846055 shutdown_connections 2024-05-08T06:37:24.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.020+0000 7fc0f2bc1640 1 -- 172.21.15.163:0/3841846055 wait complete. 2024-05-08T06:37:24.032 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T06:37:24.032 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T06:37:24.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.137+0000 7fa48a971640 1 Processor -- start 2024-05-08T06:37:24.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.138+0000 7fa48a971640 1 -- start start 2024-05-08T06:37:24.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.138+0000 7fa48a971640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4840b0db0 0x7fa484073d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:24.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.138+0000 7fa48a971640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4840b12b0 con 0x7fa4841563d0 2024-05-08T06:37:24.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.138+0000 7fa48a971640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4840b1590 con 0x7fa4840d2180 2024-05-08T06:37:24.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.138+0000 7fa48a971640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa48415fe70 con 0x7fa4840b0db0 2024-05-08T06:37:24.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa483fff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa4841563d0 0x7fa484160300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33814/0 (socket says 172.21.15.163:33814) 2024-05-08T06:37:24.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa483fff640 1 -- 172.21.15.163:0/300967966 learned_addr learned my addr 172.21.15.163:0/300967966 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:24.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 -- 172.21.15.163:0/300967966 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4173390627 0 0) 0x7fa4840b12b0 con 0x7fa4841563d0 2024-05-08T06:37:24.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 -- 172.21.15.163:0/300967966 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa46c003520 con 0x7fa4841563d0 2024-05-08T06:37:24.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa4837fe640 1 --2- 172.21.15.163:0/300967966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4840b0db0 0x7fa484073d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:24.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 -- 172.21.15.163:0/300967966 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1698537580 0 0) 0x7fa46c003520 con 0x7fa4841563d0 2024-05-08T06:37:24.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 -- 172.21.15.163:0/300967966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4840b0db0 msgr2=0x7fa484073d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:24.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 --2- 172.21.15.163:0/300967966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4840b0db0 0x7fa484073d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:24.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 -- 172.21.15.163:0/300967966 >> v1:172.21.15.192:6789/0 conn(0x7fa4840d2180 legacy=0x7fa484162fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:24.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 -- 172.21.15.163:0/300967966 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48415fb00 con 0x7fa4841563d0 2024-05-08T06:37:24.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 -- 172.21.15.163:0/300967966 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7fa4740016f0 con 0x7fa4841563d0 2024-05-08T06:37:24.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 -- 172.21.15.163:0/300967966 >> v1:172.21.15.163:6789/0 conn(0x7fa4841563d0 legacy=0x7fa484160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:24.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 --2- 172.21.15.163:0/300967966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa46c003a30 0x7fa46c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:24.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 --2- 172.21.15.163:0/300967966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4840b0db0 0x7fa46c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:24.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 --2- 172.21.15.163:0/300967966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa46c004d90 0x7fa46c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:24.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 -- 172.21.15.163:0/300967966 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa48415fe70 con 0x7fa46c004d90 2024-05-08T06:37:24.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 -- 172.21.15.163:0/300967966 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c00bbd0 con 0x7fa46c003a30 2024-05-08T06:37:24.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa482ffd640 1 -- 172.21.15.163:0/300967966 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa46c00bdd0 con 0x7fa4840b0db0 2024-05-08T06:37:24.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa488f0e640 1 --2- 172.21.15.163:0/300967966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa46c004d90 0x7fa46c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:24.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa483fff640 1 --2- 172.21.15.163:0/300967966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa46c003a30 0x7fa46c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:24.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.139+0000 7fa4837fe640 1 --2- 172.21.15.163:0/300967966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4840b0db0 0x7fa46c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:24.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.140+0000 7fa488f0e640 1 -- 172.21.15.163:0/300967966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4840b0db0 msgr2=0x7fa46c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:24.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.140+0000 7fa488f0e640 1 --2- 172.21.15.163:0/300967966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4840b0db0 0x7fa46c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:24.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.140+0000 7fa488f0e640 1 -- 172.21.15.163:0/300967966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa46c003a30 msgr2=0x7fa46c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:24.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.140+0000 7fa488f0e640 1 --2- 172.21.15.163:0/300967966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa46c003a30 0x7fa46c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:24.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.140+0000 7fa488f0e640 1 -- 172.21.15.163:0/300967966 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46c003520 con 0x7fa46c004d90 2024-05-08T06:37:24.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.140+0000 7fa488f0e640 1 --2- 172.21.15.163:0/300967966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa46c004d90 0x7fa46c005210 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa47c001e60 tx=0x7fa46c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5013a44032bb933c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:24.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.140+0000 7fa482ffd640 1 -- 172.21.15.163:0/300967966 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c001ab0 con 0x7fa46c004d90 2024-05-08T06:37:24.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.140+0000 7fa482ffd640 1 -- 172.21.15.163:0/300967966 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47c02ae10 con 0x7fa46c004d90 2024-05-08T06:37:24.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.140+0000 7fa482ffd640 1 -- 172.21.15.163:0/300967966 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c003b70 con 0x7fa46c004d90 2024-05-08T06:37:24.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.141+0000 7fa48a971640 1 -- 172.21.15.163:0/300967966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa46c004d90 msgr2=0x7fa46c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:24.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.141+0000 7fa48a971640 1 --2- 172.21.15.163:0/300967966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa46c004d90 0x7fa46c005210 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa47c001e60 tx=0x7fa46c005830 comp rx=0 tx=0).stop 2024-05-08T06:37:24.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.141+0000 7fa488f0e640 1 -- 172.21.15.163:0/300967966 reap_dead start 2024-05-08T06:37:24.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.141+0000 7fa48a971640 1 -- 172.21.15.163:0/300967966 shutdown_connections 2024-05-08T06:37:24.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.141+0000 7fa48a971640 1 -- 172.21.15.163:0/300967966 >> 172.21.15.163:0/300967966 conn(0x7fa48406d610 msgr2=0x7fa48406da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:24.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.141+0000 7fa48a971640 1 -- 172.21.15.163:0/300967966 shutdown_connections 2024-05-08T06:37:24.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.141+0000 7fa48a971640 1 -- 172.21.15.163:0/300967966 wait complete. 2024-05-08T06:37:24.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.142+0000 7fa48a971640 1 Processor -- start 2024-05-08T06:37:24.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.142+0000 7fa48a971640 1 -- start start 2024-05-08T06:37:24.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa48a971640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa46c004d90 0x7fa484158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:24.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa48a971640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa484159290 0x7fa484145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:24.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa48a971640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa484151b70 0x7fa4840b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:24.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa48a971640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa484156930 con 0x7fa46c004d90 2024-05-08T06:37:24.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa48a971640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841567b0 con 0x7fa484159290 2024-05-08T06:37:24.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa48a971640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa4841636f0 con 0x7fa484151b70 2024-05-08T06:37:24.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa483fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa46c004d90 0x7fa484158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:24.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa483fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa46c004d90 0x7fa484158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53532/0 (socket says 172.21.15.163:53532) 2024-05-08T06:37:24.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa483fff640 1 -- 172.21.15.163:0/877468609 learned_addr learned my addr 172.21.15.163:0/877468609 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:24.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa488f0e640 1 --2- 172.21.15.163:0/877468609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa484151b70 0x7fa4840b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:24.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa4837fe640 1 --2- 172.21.15.163:0/877468609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa484159290 0x7fa484145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:24.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa483fff640 1 -- 172.21.15.163:0/877468609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa484151b70 msgr2=0x7fa4840b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:24.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa483fff640 1 --2- 172.21.15.163:0/877468609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa484151b70 0x7fa4840b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:24.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa483fff640 1 -- 172.21.15.163:0/877468609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa484159290 msgr2=0x7fa484145910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:24.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa483fff640 1 --2- 172.21.15.163:0/877468609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa484159290 0x7fa484145910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:24.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.143+0000 7fa483fff640 1 -- 172.21.15.163:0/877468609 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa484159a70 con 0x7fa46c004d90 2024-05-08T06:37:24.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.144+0000 7fa483fff640 1 --2- 172.21.15.163:0/877468609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa46c004d90 0x7fa484158110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa474007290 tx=0x7fa474007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:24.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.144+0000 7fa480ff9640 1 -- 172.21.15.163:0/877468609 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47400f9c0 con 0x7fa46c004d90 2024-05-08T06:37:24.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.144+0000 7fa480ff9640 1 -- 172.21.15.163:0/877468609 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa474018070 con 0x7fa46c004d90 2024-05-08T06:37:24.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.144+0000 7fa48a971640 1 -- 172.21.15.163:0/877468609 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa484145e50 con 0x7fa46c004d90 2024-05-08T06:37:24.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.144+0000 7fa480ff9640 1 -- 172.21.15.163:0/877468609 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47400fdc0 con 0x7fa46c004d90 2024-05-08T06:37:24.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.144+0000 7fa48a971640 1 -- 172.21.15.163:0/877468609 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa484157630 con 0x7fa46c004d90 2024-05-08T06:37:24.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.145+0000 7fa48a971640 1 -- 172.21.15.163:0/877468609 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48406a550 con 0x7fa46c004d90 2024-05-08T06:37:24.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.145+0000 7fa480ff9640 1 -- 172.21.15.163:0/877468609 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa47401d460 con 0x7fa46c004d90 2024-05-08T06:37:24.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.146+0000 7fa480ff9640 1 --2- 172.21.15.163:0/877468609 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa4700762d0 0x7fa470078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:24.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.146+0000 7fa4837fe640 1 --2- 172.21.15.163:0/877468609 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa4700762d0 0x7fa470078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:24.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.146+0000 7fa480ff9640 1 -- 172.21.15.163:0/877468609 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fa474097630 con 0x7fa46c004d90 2024-05-08T06:37:24.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.146+0000 7fa4837fe640 1 --2- 172.21.15.163:0/877468609 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa4700762d0 0x7fa470078790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa484155e80 tx=0x7fa478008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:24.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.150+0000 7fa480ff9640 1 -- 172.21.15.163:0/877468609 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa474060f90 con 0x7fa46c004d90 2024-05-08T06:37:24.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:24.367+0000 7fa48a971640 1 -- 172.21.15.163:0/877468609 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa48406e440 con 0x7fa46c004d90 2024-05-08T06:37:25.025 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.026+0000 7fa480ff9640 1 -- 172.21.15.163:0/877468609 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fa474060890 con 0x7fa46c004d90 2024-05-08T06:37:25.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.028+0000 7fa48a971640 1 -- 172.21.15.163:0/877468609 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa4700762d0 msgr2=0x7fa470078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.028+0000 7fa48a971640 1 --2- 172.21.15.163:0/877468609 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa4700762d0 0x7fa470078790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa484155e80 tx=0x7fa478008040 comp rx=0 tx=0).stop 2024-05-08T06:37:25.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.028+0000 7fa48a971640 1 -- 172.21.15.163:0/877468609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa46c004d90 msgr2=0x7fa484158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.028+0000 7fa48a971640 1 --2- 172.21.15.163:0/877468609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa46c004d90 0x7fa484158110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa474007290 tx=0x7fa474007750 comp rx=0 tx=0).stop 2024-05-08T06:37:25.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.028+0000 7fa48a971640 1 -- 172.21.15.163:0/877468609 shutdown_connections 2024-05-08T06:37:25.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.028+0000 7fa48a971640 1 --2- 172.21.15.163:0/877468609 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa4700762d0 0x7fa470078790 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.028+0000 7fa48a971640 1 --2- 172.21.15.163:0/877468609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa484151b70 0x7fa4840b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.028+0000 7fa48a971640 1 --2- 172.21.15.163:0/877468609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa484159290 0x7fa484145910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.028+0000 7fa48a971640 1 --2- 172.21.15.163:0/877468609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa46c004d90 0x7fa484158110 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.028+0000 7fa48a971640 1 -- 172.21.15.163:0/877468609 >> 172.21.15.163:0/877468609 conn(0x7fa48406d610 msgr2=0x7fa484070360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:25.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.028+0000 7fa48a971640 1 -- 172.21.15.163:0/877468609 shutdown_connections 2024-05-08T06:37:25.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.028+0000 7fa48a971640 1 -- 172.21.15.163:0/877468609 wait complete. 2024-05-08T06:37:25.040 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T06:37:25.040 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:25.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.144+0000 7fc8b461c640 1 Processor -- start 2024-05-08T06:37:25.143 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.144+0000 7fc8b461c640 1 -- start start 2024-05-08T06:37:25.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b461c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8ac15d600 0x7fc8ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.146 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b461c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc8ac06d870 0x7fc8ac06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b461c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8ac06f410 con 0x7fc8ac06ef80 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b461c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc8ac06dcb0 con 0x7fc8ac06d870 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b461c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc8ac06f6c0 con 0x7fc8ac15d600 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b2bb9640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc8ac06ef80 0x7fc8ac0ce2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33822/0 (socket says 172.21.15.163:33822) 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b2bb9640 1 -- 172.21.15.163:0/4292727381 learned_addr learned my addr 172.21.15.163:0/4292727381 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b23b8640 1 --2- 172.21.15.163:0/4292727381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8ac15d600 0x7fc8ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b13b6640 1 -- 172.21.15.163:0/4292727381 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1704834230 0 0) 0x7fc8ac06f410 con 0x7fc8ac06ef80 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b1bb7640 1 --2- 172.21.15.163:0/4292727381 >> v2:172.21.15.192:3300/0 conn(0x7fc8ac06d870 0x7fc8ac06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b13b6640 1 -- 172.21.15.163:0/4292727381 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc898003520 con 0x7fc8ac06ef80 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b13b6640 1 -- 172.21.15.163:0/4292727381 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2120094397 0 0) 0x7fc898003520 con 0x7fc8ac06ef80 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b13b6640 1 -- 172.21.15.163:0/4292727381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8ac15d600 msgr2=0x7fc8ac06d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b13b6640 1 --2- 172.21.15.163:0/4292727381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8ac15d600 0x7fc8ac06d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b13b6640 1 -- 172.21.15.163:0/4292727381 >> v2:172.21.15.192:3300/0 conn(0x7fc8ac06d870 msgr2=0x7fc8ac06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b13b6640 1 --2- 172.21.15.163:0/4292727381 >> v2:172.21.15.192:3300/0 conn(0x7fc8ac06d870 0x7fc8ac06ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.147 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b13b6640 1 -- 172.21.15.163:0/4292727381 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ac15de20 con 0x7fc8ac06ef80 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.145+0000 7fc8b13b6640 1 -- 172.21.15.163:0/4292727381 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7fc8a40016f0 con 0x7fc8ac06ef80 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b13b6640 1 -- 172.21.15.163:0/4292727381 >> v1:172.21.15.163:6789/0 conn(0x7fc8ac06ef80 legacy=0x7fc8ac0ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b13b6640 1 --2- 172.21.15.163:0/4292727381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc898003a00 0x7fc898003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b13b6640 1 --2- 172.21.15.163:0/4292727381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc898004800 0x7fc898004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b13b6640 1 --2- 172.21.15.163:0/4292727381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8ac15d600 0x7fc8980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b13b6640 1 -- 172.21.15.163:0/4292727381 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac06dcb0 con 0x7fc898004800 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b13b6640 1 -- 172.21.15.163:0/4292727381 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac06f6c0 con 0x7fc898003a00 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b13b6640 1 -- 172.21.15.163:0/4292727381 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc89800ba40 con 0x7fc8ac15d600 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b1bb7640 1 --2- 172.21.15.163:0/4292727381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc898004800 0x7fc898004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b23b8640 1 --2- 172.21.15.163:0/4292727381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc898003a00 0x7fc898003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b2bb9640 1 --2- 172.21.15.163:0/4292727381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8ac15d600 0x7fc8980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b1bb7640 1 -- 172.21.15.163:0/4292727381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8ac15d600 msgr2=0x7fc8980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b1bb7640 1 --2- 172.21.15.163:0/4292727381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8ac15d600 0x7fc8980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b1bb7640 1 -- 172.21.15.163:0/4292727381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc898003a00 msgr2=0x7fc898003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:25.148 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b1bb7640 1 --2- 172.21.15.163:0/4292727381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc898003a00 0x7fc898003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b1bb7640 1 -- 172.21.15.163:0/4292727381 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc898003520 con 0x7fc898004800 2024-05-08T06:37:25.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.146+0000 7fc8b2bb9640 1 --2- 172.21.15.163:0/4292727381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8ac15d600 0x7fc8980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:25.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.147+0000 7fc8b1bb7640 1 --2- 172.21.15.163:0/4292727381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc898004800 0x7fc898004c80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009ac0 tx=0x7fc898005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0db200d40c0ef01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:25.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.147+0000 7fc8b13b6640 1 -- 172.21.15.163:0/4292727381 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c036070 con 0x7fc898004800 2024-05-08T06:37:25.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.147+0000 7fc8b13b6640 1 -- 172.21.15.163:0/4292727381 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc89c031400 con 0x7fc898004800 2024-05-08T06:37:25.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.147+0000 7fc8b13b6640 1 -- 172.21.15.163:0/4292727381 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c009690 con 0x7fc898004800 2024-05-08T06:37:25.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.147+0000 7fc8b461c640 1 -- 172.21.15.163:0/4292727381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc898004800 msgr2=0x7fc898004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.147+0000 7fc8b461c640 1 --2- 172.21.15.163:0/4292727381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc898004800 0x7fc898004c80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009ac0 tx=0x7fc898005830 comp rx=0 tx=0).stop 2024-05-08T06:37:25.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.147+0000 7fc8b2bb9640 1 -- 172.21.15.163:0/4292727381 reap_dead start 2024-05-08T06:37:25.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.147+0000 7fc8b461c640 1 -- 172.21.15.163:0/4292727381 shutdown_connections 2024-05-08T06:37:25.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.147+0000 7fc8b461c640 1 -- 172.21.15.163:0/4292727381 >> 172.21.15.163:0/4292727381 conn(0x7fc8ac072ba0 msgr2=0x7fc8ac072f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:25.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.147+0000 7fc8b461c640 1 -- 172.21.15.163:0/4292727381 shutdown_connections 2024-05-08T06:37:25.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.147+0000 7fc8b461c640 1 -- 172.21.15.163:0/4292727381 wait complete. 2024-05-08T06:37:25.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.149+0000 7fc8b461c640 1 Processor -- start 2024-05-08T06:37:25.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.149+0000 7fc8b461c640 1 -- start start 2024-05-08T06:37:25.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.149+0000 7fc8b461c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc898004800 0x7fc8ac13ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.149+0000 7fc8b461c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8ac15fc30 0x7fc8ac160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.149+0000 7fc8b461c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8ac1f66f0 0x7fc8ac1f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.149+0000 7fc8b461c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac06f9a0 con 0x7fc8ac15fc30 2024-05-08T06:37:25.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.149+0000 7fc8b461c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac15e3e0 con 0x7fc898004800 2024-05-08T06:37:25.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.149+0000 7fc8b461c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc8ac15d9e0 con 0x7fc8ac1f66f0 2024-05-08T06:37:25.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.149+0000 7fc8b1bb7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8ac15fc30 0x7fc8ac160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.149+0000 7fc8b1bb7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8ac15fc30 0x7fc8ac160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53546/0 (socket says 172.21.15.163:53546) 2024-05-08T06:37:25.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.149+0000 7fc8b1bb7640 1 -- 172.21.15.163:0/1082949411 learned_addr learned my addr 172.21.15.163:0/1082949411 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.150+0000 7fc8b1bb7640 1 -- 172.21.15.163:0/1082949411 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8ac1f66f0 msgr2=0x7fc8ac1f6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.150+0000 7fc8b1bb7640 1 --2- 172.21.15.163:0/1082949411 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8ac1f66f0 0x7fc8ac1f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.150+0000 7fc8b1bb7640 1 -- 172.21.15.163:0/1082949411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc898004800 msgr2=0x7fc8ac13ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.150+0000 7fc8b1bb7640 1 --2- 172.21.15.163:0/1082949411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc898004800 0x7fc8ac13ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.150+0000 7fc8b1bb7640 1 -- 172.21.15.163:0/1082949411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ac1427a0 con 0x7fc8ac15fc30 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.150+0000 7fc8b1bb7640 1 --2- 172.21.15.163:0/1082949411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8ac15fc30 0x7fc8ac160090 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009be0 tx=0x7fc89c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.150+0000 7fc8a2ffd640 1 -- 172.21.15.163:0/1082949411 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c040020 con 0x7fc8ac15fc30 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.150+0000 7fc8a2ffd640 1 -- 172.21.15.163:0/1082949411 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc89c0038d0 con 0x7fc8ac15fc30 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.150+0000 7fc8b461c640 1 -- 172.21.15.163:0/1082949411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ac13f4c0 con 0x7fc8ac15fc30 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.150+0000 7fc8a2ffd640 1 -- 172.21.15.163:0/1082949411 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c036040 con 0x7fc8ac15fc30 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.150+0000 7fc8b461c640 1 -- 172.21.15.163:0/1082949411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ac141cc0 con 0x7fc8ac15fc30 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.151+0000 7fc8b461c640 1 -- 172.21.15.163:0/1082949411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ac06a550 con 0x7fc8ac15fc30 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.152+0000 7fc8a2ffd640 1 -- 172.21.15.163:0/1082949411 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc89c043020 con 0x7fc8ac15fc30 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.152+0000 7fc8a2ffd640 1 --2- 172.21.15.163:0/1082949411 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc8900762d0 0x7fc890078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.152+0000 7fc8b23b8640 1 --2- 172.21.15.163:0/1082949411 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc8900762d0 0x7fc890078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.152+0000 7fc8a2ffd640 1 -- 172.21.15.163:0/1082949411 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fc89c0b76c0 con 0x7fc8ac15fc30 2024-05-08T06:37:25.154 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.153+0000 7fc8b23b8640 1 --2- 172.21.15.163:0/1082949411 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc8900762d0 0x7fc890078790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009ee0 tx=0x7fc8a8009750 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:25.155 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.156+0000 7fc8a2ffd640 1 -- 172.21.15.163:0/1082949411 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc89c04a070 con 0x7fc8ac15fc30 2024-05-08T06:37:25.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.375+0000 7fc8b461c640 1 -- 172.21.15.163:0/1082949411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8ac0cf780 con 0x7fc8ac15fc30 2024-05-08T06:37:25.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.375+0000 7fc8a2ffd640 1 -- 172.21.15.163:0/1082949411 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5190 (secure 0 0 0) 0x7fc89c0810b0 con 0x7fc8ac15fc30 2024-05-08T06:37:25.375 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:25.375 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1773142301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1773142301},{"type":"v1","addr":"172.21.15.163:6835","nonce":1773142301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2952930735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2952930735},{"type":"v1","addr":"172.21.15.192:6834","nonce":2952930735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1297249355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1297249355},{"type":"v1","addr":"172.21.15.192:6835","nonce":1297249355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:37:22.365453+0000","modified":"2024-05-08T06:37:24.368498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4732":{"gid":4732,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/880835577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":880835577},{"type":"v1","addr":"172.21.15.163:6837","nonce":880835577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4732,"qdb_cluster":[4732]},"id":2}]} 2024-05-08T06:37:25.375 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 14 2024-05-08T06:37:25.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.377+0000 7fc8b461c640 1 -- 172.21.15.163:0/1082949411 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc8900762d0 msgr2=0x7fc890078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.377+0000 7fc8b461c640 1 --2- 172.21.15.163:0/1082949411 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc8900762d0 0x7fc890078790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009ee0 tx=0x7fc8a8009750 comp rx=0 tx=0).stop 2024-05-08T06:37:25.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.377+0000 7fc8b461c640 1 -- 172.21.15.163:0/1082949411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8ac15fc30 msgr2=0x7fc8ac160090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.377+0000 7fc8b461c640 1 --2- 172.21.15.163:0/1082949411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8ac15fc30 0x7fc8ac160090 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009be0 tx=0x7fc89c004730 comp rx=0 tx=0).stop 2024-05-08T06:37:25.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.377+0000 7fc8b461c640 1 -- 172.21.15.163:0/1082949411 shutdown_connections 2024-05-08T06:37:25.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.377+0000 7fc8b461c640 1 --2- 172.21.15.163:0/1082949411 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc8900762d0 0x7fc890078790 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.377+0000 7fc8b461c640 1 --2- 172.21.15.163:0/1082949411 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8ac1f66f0 0x7fc8ac1f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.377+0000 7fc8b461c640 1 --2- 172.21.15.163:0/1082949411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8ac15fc30 0x7fc8ac160090 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.377+0000 7fc8b461c640 1 --2- 172.21.15.163:0/1082949411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc898004800 0x7fc8ac13ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.377+0000 7fc8b461c640 1 -- 172.21.15.163:0/1082949411 >> 172.21.15.163:0/1082949411 conn(0x7fc8ac072ba0 msgr2=0x7fc8ac0cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:25.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.378+0000 7fc8b461c640 1 -- 172.21.15.163:0/1082949411 shutdown_connections 2024-05-08T06:37:25.378 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.378+0000 7fc8b461c640 1 -- 172.21.15.163:0/1082949411 wait complete. 2024-05-08T06:37:25.388 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:37:25.488 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.489+0000 7fa0fd6a9640 1 Processor -- start 2024-05-08T06:37:25.488 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.489+0000 7fa0fd6a9640 1 -- start start 2024-05-08T06:37:25.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.490+0000 7fa0fd6a9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa0f8142390 0x7fa0f80b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.490+0000 7fa0fd6a9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0f80b65e0 0x7fa0f8162fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.490+0000 7fa0fd6a9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0f8142770 con 0x7fa0f8163500 2024-05-08T06:37:25.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.490+0000 7fa0fd6a9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa0f8143a80 con 0x7fa0f80b65e0 2024-05-08T06:37:25.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.490+0000 7fa0fd6a9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa0f8163d30 con 0x7fa0f8142390 2024-05-08T06:37:25.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.490+0000 7fa0ee7fc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0f80b65e0 0x7fa0f8162fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.490+0000 7fa0ee7fc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0f80b65e0 0x7fa0f8162fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53560/0 (socket says 172.21.15.163:53560) 2024-05-08T06:37:25.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.490+0000 7fa0ee7fc640 1 -- 172.21.15.163:0/161426413 learned_addr learned my addr 172.21.15.163:0/161426413 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:25.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.490+0000 7fa0f6ffd640 1 --2- 172.21.15.163:0/161426413 >> v2:172.21.15.192:3300/0 conn(0x7fa0f8142390 0x7fa0f80b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.490+0000 7fa0ee7fc640 1 -- 172.21.15.163:0/161426413 >> v2:172.21.15.192:3300/0 conn(0x7fa0f8142390 msgr2=0x7fa0f80b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:25.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.490+0000 7fa0ee7fc640 1 --2- 172.21.15.163:0/161426413 >> v2:172.21.15.192:3300/0 conn(0x7fa0f8142390 0x7fa0f80b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.490+0000 7fa0ee7fc640 1 -- 172.21.15.163:0/161426413 >> v1:172.21.15.192:6789/0 conn(0x7fa0f8163500 legacy=0x7fa0f8071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:25.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.490+0000 7fa0ee7fc640 1 -- 172.21.15.163:0/161426413 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f815f990 con 0x7fa0f80b65e0 2024-05-08T06:37:25.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.490+0000 7fa0f6ffd640 1 --2- 172.21.15.163:0/161426413 >> v2:172.21.15.192:3300/0 conn(0x7fa0f8142390 0x7fa0f80b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:25.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.491+0000 7fa0ee7fc640 1 --2- 172.21.15.163:0/161426413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0f80b65e0 0x7fa0f8162fc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80b8790 tx=0x7fa0e002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2156fbdd262f8b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:25.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.491+0000 7fa0f67fc640 1 -- 172.21.15.163:0/161426413 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0037070 con 0x7fa0f80b65e0 2024-05-08T06:37:25.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.491+0000 7fa0f67fc640 1 -- 172.21.15.163:0/161426413 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e0032440 con 0x7fa0f80b65e0 2024-05-08T06:37:25.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.491+0000 7fa0f67fc640 1 -- 172.21.15.163:0/161426413 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0032810 con 0x7fa0f80b65e0 2024-05-08T06:37:25.494 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.492+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/161426413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0f80b65e0 msgr2=0x7fa0f8162fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.494 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.492+0000 7fa0fd6a9640 1 --2- 172.21.15.163:0/161426413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0f80b65e0 0x7fa0f8162fc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80b8790 tx=0x7fa0e002a1e0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.492+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/161426413 shutdown_connections 2024-05-08T06:37:25.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.492+0000 7fa0fd6a9640 1 --2- 172.21.15.163:0/161426413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0f80b65e0 0x7fa0f8162fc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.492+0000 7fa0fd6a9640 1 --2- 172.21.15.163:0/161426413 >> v2:172.21.15.192:3300/0 conn(0x7fa0f8142390 0x7fa0f80b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.492+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/161426413 >> 172.21.15.163:0/161426413 conn(0x7fa0f80b4da0 msgr2=0x7fa0f80b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:25.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.492+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/161426413 shutdown_connections 2024-05-08T06:37:25.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.492+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/161426413 wait complete. 2024-05-08T06:37:25.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.493+0000 7fa0fd6a9640 1 Processor -- start 2024-05-08T06:37:25.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.493+0000 7fa0fd6a9640 1 -- start start 2024-05-08T06:37:25.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0fd6a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0f80b65e0 0x7fa0f806e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0fd6a9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f8142390 0x7fa0f8162580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0fd6a9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0f8163500 0x7fa0f81f9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0fd6a9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f8160230 con 0x7fa0f8163500 2024-05-08T06:37:25.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0fd6a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f815ff50 con 0x7fa0f80b65e0 2024-05-08T06:37:25.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0fd6a9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f8160750 con 0x7fa0f8142390 2024-05-08T06:37:25.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0f77fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0f8163500 0x7fa0f81f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0f6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0f80b65e0 0x7fa0f806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0f77fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0f8163500 0x7fa0f81f9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53566/0 (socket says 172.21.15.163:53566) 2024-05-08T06:37:25.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0f77fe640 1 -- 172.21.15.163:0/2731645530 learned_addr learned my addr 172.21.15.163:0/2731645530 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:25.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0ee7fc640 1 --2- 172.21.15.163:0/2731645530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f8142390 0x7fa0f8162580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0f77fe640 1 -- 172.21.15.163:0/2731645530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f8142390 msgr2=0x7fa0f8162580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0f77fe640 1 --2- 172.21.15.163:0/2731645530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f8142390 0x7fa0f8162580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0f77fe640 1 -- 172.21.15.163:0/2731645530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0f80b65e0 msgr2=0x7fa0f806e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0f77fe640 1 --2- 172.21.15.163:0/2731645530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0f80b65e0 0x7fa0f806e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.494+0000 7fa0f77fe640 1 -- 172.21.15.163:0/2731645530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f806ece0 con 0x7fa0f8163500 2024-05-08T06:37:25.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.495+0000 7fa0f77fe640 1 --2- 172.21.15.163:0/2731645530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0f8163500 0x7fa0f81f9760 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000b9a0 tx=0x7fa0f000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:25.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.495+0000 7fa0effff640 1 -- 172.21.15.163:0/2731645530 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0019070 con 0x7fa0f8163500 2024-05-08T06:37:25.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.495+0000 7fa0effff640 1 -- 172.21.15.163:0/2731645530 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f000cba0 con 0x7fa0f8163500 2024-05-08T06:37:25.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.495+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/2731645530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f806efc0 con 0x7fa0f8163500 2024-05-08T06:37:25.497 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.495+0000 7fa0effff640 1 -- 172.21.15.163:0/2731645530 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0014440 con 0x7fa0f8163500 2024-05-08T06:37:25.497 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.495+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/2731645530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f813c400 con 0x7fa0f8163500 2024-05-08T06:37:25.497 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.496+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/2731645530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f8075dc0 con 0x7fa0f8163500 2024-05-08T06:37:25.497 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.496+0000 7fa0effff640 1 -- 172.21.15.163:0/2731645530 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa0f001c070 con 0x7fa0f8163500 2024-05-08T06:37:25.497 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.497+0000 7fa0effff640 1 --2- 172.21.15.163:0/2731645530 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa0d80762d0 0x7fa0d8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.497 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.497+0000 7fa0effff640 1 -- 172.21.15.163:0/2731645530 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fa0f0097240 con 0x7fa0f8163500 2024-05-08T06:37:25.498 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.497+0000 7fa0f6ffd640 1 --2- 172.21.15.163:0/2731645530 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa0d80762d0 0x7fa0d8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.498 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.497+0000 7fa0f6ffd640 1 --2- 172.21.15.163:0/2731645530 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa0d80762d0 0x7fa0d8078790 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80705e0 tx=0x7fa0e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:25.500 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.501+0000 7fa0effff640 1 -- 172.21.15.163:0/2731645530 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa0f0060550 con 0x7fa0f8163500 2024-05-08T06:37:25.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.681+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/2731645530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0f815d260 con 0x7fa0f8163500 2024-05-08T06:37:25.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.682+0000 7fa0effff640 1 -- 172.21.15.163:0/2731645530 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16693 (secure 0 0 0) 0x7fa0f815d260 con 0x7fa0f8163500 2024-05-08T06:37:25.681 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:25.681 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":31,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:37:24.017870+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:37:20.131664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T06:37:21.318802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:37:25.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.684+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/2731645530 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa0d80762d0 msgr2=0x7fa0d8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.684+0000 7fa0fd6a9640 1 --2- 172.21.15.163:0/2731645530 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa0d80762d0 0x7fa0d8078790 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80705e0 tx=0x7fa0e8008040 comp rx=0 tx=0).stop 2024-05-08T06:37:25.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.684+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/2731645530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0f8163500 msgr2=0x7fa0f81f9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.684+0000 7fa0fd6a9640 1 --2- 172.21.15.163:0/2731645530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0f8163500 0x7fa0f81f9760 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000b9a0 tx=0x7fa0f000be60 comp rx=0 tx=0).stop 2024-05-08T06:37:25.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.684+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/2731645530 shutdown_connections 2024-05-08T06:37:25.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.684+0000 7fa0fd6a9640 1 --2- 172.21.15.163:0/2731645530 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa0d80762d0 0x7fa0d8078790 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.684+0000 7fa0fd6a9640 1 --2- 172.21.15.163:0/2731645530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0f8163500 0x7fa0f81f9760 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.684+0000 7fa0fd6a9640 1 --2- 172.21.15.163:0/2731645530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f8142390 0x7fa0f8162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.684+0000 7fa0fd6a9640 1 --2- 172.21.15.163:0/2731645530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0f80b65e0 0x7fa0f806e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.684+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/2731645530 >> 172.21.15.163:0/2731645530 conn(0x7fa0f80b4da0 msgr2=0x7fa0f8143c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:25.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.684+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/2731645530 shutdown_connections 2024-05-08T06:37:25.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.684+0000 7fa0fd6a9640 1 -- 172.21.15.163:0/2731645530 wait complete. 2024-05-08T06:37:25.695 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T06:37:25.695 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:37:25.799 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.800+0000 7ff3a6e93640 1 Processor -- start 2024-05-08T06:37:25.799 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.800+0000 7ff3a6e93640 1 -- start start 2024-05-08T06:37:25.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.801+0000 7ff3a6e93640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff3a00ae180 0x7ff3a00b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.801+0000 7ff3a6e93640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3a0072f80 con 0x7ff3a00b46e0 2024-05-08T06:37:25.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.801+0000 7ff3a6e93640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3a0074430 con 0x7ff3a015c020 2024-05-08T06:37:25.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.801+0000 7ff3a6e93640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff3a00ae700 con 0x7ff3a00ae180 2024-05-08T06:37:25.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.801+0000 7ff3a5430640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff3a00b46e0 0x7ff3a00afc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33826/0 (socket says 172.21.15.163:33826) 2024-05-08T06:37:25.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.801+0000 7ff3a5430640 1 -- 172.21.15.163:0/684326529 learned_addr learned my addr 172.21.15.163:0/684326529 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:25.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.801+0000 7ff39ffff640 1 --2- 172.21.15.163:0/684326529 >> v2:172.21.15.192:3300/0 conn(0x7ff3a00ae180 0x7ff3a00b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.801+0000 7ff39f7fe640 1 -- 172.21.15.163:0/684326529 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 848030457 0 0) 0x7ff398001110 con 0x7ff3a00b46e0 2024-05-08T06:37:25.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.801+0000 7ff39f7fe640 1 -- 172.21.15.163:0/684326529 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff38c003520 con 0x7ff3a00b46e0 2024-05-08T06:37:25.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.801+0000 7ff39ffff640 1 -- 172.21.15.163:0/684326529 >> v1:172.21.15.192:6789/0 conn(0x7ff3a015c020 legacy=0x7ff3a0073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.801+0000 7ff39ffff640 1 -- 172.21.15.163:0/684326529 >> v1:172.21.15.163:6789/0 conn(0x7ff3a00b46e0 legacy=0x7ff3a00afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.801+0000 7ff39ffff640 1 -- 172.21.15.163:0/684326529 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a0074710 con 0x7ff3a00ae180 2024-05-08T06:37:25.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.802+0000 7ff39ffff640 1 --2- 172.21.15.163:0/684326529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3a00ae180 0x7ff3a00b41a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff3880097a0 tx=0x7ff38802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c6d98b76cd4bc6a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:25.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.802+0000 7ff39f7fe640 1 -- 172.21.15.163:0/684326529 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388037070 con 0x7ff3a00ae180 2024-05-08T06:37:25.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.802+0000 7ff39f7fe640 1 -- 172.21.15.163:0/684326529 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff388032440 con 0x7ff3a00ae180 2024-05-08T06:37:25.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.802+0000 7ff39f7fe640 1 -- 172.21.15.163:0/684326529 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38802ae60 con 0x7ff3a00ae180 2024-05-08T06:37:25.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.803+0000 7ff3a6e93640 1 -- 172.21.15.163:0/684326529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3a00ae180 msgr2=0x7ff3a00b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.803+0000 7ff3a6e93640 1 --2- 172.21.15.163:0/684326529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3a00ae180 0x7ff3a00b41a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff3880097a0 tx=0x7ff38802a110 comp rx=0 tx=0).stop 2024-05-08T06:37:25.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.803+0000 7ff3a6e93640 1 -- 172.21.15.163:0/684326529 shutdown_connections 2024-05-08T06:37:25.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.803+0000 7ff3a6e93640 1 --2- 172.21.15.163:0/684326529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3a00ae180 0x7ff3a00b41a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.803+0000 7ff3a6e93640 1 -- 172.21.15.163:0/684326529 >> 172.21.15.163:0/684326529 conn(0x7ff3a006c500 msgr2=0x7ff3a006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:25.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.803+0000 7ff3a6e93640 1 -- 172.21.15.163:0/684326529 shutdown_connections 2024-05-08T06:37:25.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.803+0000 7ff3a6e93640 1 -- 172.21.15.163:0/684326529 wait complete. 2024-05-08T06:37:25.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.804+0000 7ff3a6e93640 1 Processor -- start 2024-05-08T06:37:25.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.804+0000 7ff3a6e93640 1 -- start start 2024-05-08T06:37:25.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a6e93640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3a00ae180 0x7ff3a00b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a6e93640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3a00b46e0 0x7ff3a00b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a6e93640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a015c020 0x7ff3a01f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a6e93640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a015c400 con 0x7ff3a015c020 2024-05-08T06:37:25.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a6e93640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a0073290 con 0x7ff3a00ae180 2024-05-08T06:37:25.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a6e93640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a00b4b70 con 0x7ff3a00b46e0 2024-05-08T06:37:25.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a5430640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a015c020 0x7ff3a01f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a5430640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a015c020 0x7ff3a01f26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53578/0 (socket says 172.21.15.163:53578) 2024-05-08T06:37:25.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a5430640 1 -- 172.21.15.163:0/1928487180 learned_addr learned my addr 172.21.15.163:0/1928487180 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:25.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a4c2f640 1 --2- 172.21.15.163:0/1928487180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3a00ae180 0x7ff3a00b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff39ffff640 1 --2- 172.21.15.163:0/1928487180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3a00b46e0 0x7ff3a00b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a5430640 1 -- 172.21.15.163:0/1928487180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3a00b46e0 msgr2=0x7ff3a00b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a5430640 1 --2- 172.21.15.163:0/1928487180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3a00b46e0 0x7ff3a00b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a5430640 1 -- 172.21.15.163:0/1928487180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3a00ae180 msgr2=0x7ff3a00b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:25.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a5430640 1 --2- 172.21.15.163:0/1928487180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3a00ae180 0x7ff3a00b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:25.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a5430640 1 -- 172.21.15.163:0/1928487180 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a00b2160 con 0x7ff3a015c020 2024-05-08T06:37:25.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a5430640 1 --2- 172.21.15.163:0/1928487180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a015c020 0x7ff3a01f26a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff39800c700 tx=0x7ff39800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:25.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff39d7fa640 1 -- 172.21.15.163:0/1928487180 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3980058b0 con 0x7ff3a015c020 2024-05-08T06:37:25.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff39d7fa640 1 -- 172.21.15.163:0/1928487180 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff398005a50 con 0x7ff3a015c020 2024-05-08T06:37:25.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff39d7fa640 1 -- 172.21.15.163:0/1928487180 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff398005e40 con 0x7ff3a015c020 2024-05-08T06:37:25.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.805+0000 7ff3a6e93640 1 -- 172.21.15.163:0/1928487180 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a00b2440 con 0x7ff3a015c020 2024-05-08T06:37:25.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.806+0000 7ff3a6e93640 1 -- 172.21.15.163:0/1928487180 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a00b2a00 con 0x7ff3a015c020 2024-05-08T06:37:25.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.806+0000 7ff3a6e93640 1 -- 172.21.15.163:0/1928487180 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a006a550 con 0x7ff3a015c020 2024-05-08T06:37:25.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.807+0000 7ff39d7fa640 1 -- 172.21.15.163:0/1928487180 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff398007280 con 0x7ff3a015c020 2024-05-08T06:37:25.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.807+0000 7ff39d7fa640 1 --2- 172.21.15.163:0/1928487180 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff3800762d0 0x7ff380078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:25.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.808+0000 7ff3a4c2f640 1 --2- 172.21.15.163:0/1928487180 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff3800762d0 0x7ff380078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:25.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.808+0000 7ff39d7fa640 1 -- 172.21.15.163:0/1928487180 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7ff3980972b0 con 0x7ff3a015c020 2024-05-08T06:37:25.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.808+0000 7ff3a4c2f640 1 --2- 172.21.15.163:0/1928487180 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff3800762d0 0x7ff380078790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff390000f80 tx=0x7ff390009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:25.811 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.812+0000 7ff39d7fa640 1 -- 172.21.15.163:0/1928487180 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff3a0072f80 con 0x7ff3a015c020 2024-05-08T06:37:25.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.992+0000 7ff3a6e93640 1 -- 172.21.15.163:0/1928487180 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3a01f50a0 con 0x7ff3800762d0 2024-05-08T06:37:25.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:25.998+0000 7ff39d7fa640 1 -- 172.21.15.163:0/1928487180 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237184 (secure 0 0 0) 0x7ff3a01f50a0 con 0x7ff3800762d0 2024-05-08T06:37:26.001 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:26.002 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2024-05-08T06:37:26.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.001+0000 7ff3a6e93640 1 -- 172.21.15.163:0/1928487180 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff3800762d0 msgr2=0x7ff380078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.001+0000 7ff3a6e93640 1 --2- 172.21.15.163:0/1928487180 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff3800762d0 0x7ff380078790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff390000f80 tx=0x7ff390009040 comp rx=0 tx=0).stop 2024-05-08T06:37:26.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.001+0000 7ff3a6e93640 1 -- 172.21.15.163:0/1928487180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a015c020 msgr2=0x7ff3a01f26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.001+0000 7ff3a6e93640 1 --2- 172.21.15.163:0/1928487180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a015c020 0x7ff3a01f26a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff39800c700 tx=0x7ff39800cbc0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.001+0000 7ff3a6e93640 1 -- 172.21.15.163:0/1928487180 shutdown_connections 2024-05-08T06:37:26.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.001+0000 7ff3a6e93640 1 --2- 172.21.15.163:0/1928487180 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff3800762d0 0x7ff380078790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.001+0000 7ff3a6e93640 1 --2- 172.21.15.163:0/1928487180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a015c020 0x7ff3a01f26a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.001+0000 7ff3a6e93640 1 --2- 172.21.15.163:0/1928487180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3a00b46e0 0x7ff3a00b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.001+0000 7ff3a6e93640 1 --2- 172.21.15.163:0/1928487180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3a00ae180 0x7ff3a00b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.001+0000 7ff3a6e93640 1 -- 172.21.15.163:0/1928487180 >> 172.21.15.163:0/1928487180 conn(0x7ff3a006c500 msgr2=0x7ff3a015b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:26.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.001+0000 7ff3a6e93640 1 -- 172.21.15.163:0/1928487180 shutdown_connections 2024-05-08T06:37:26.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.001+0000 7ff3a6e93640 1 -- 172.21.15.163:0/1928487180 wait complete. 2024-05-08T06:37:26.012 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-05-08T06:37:24.106018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219728,"kb_used_data":3496,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754992,"statfs":{"total":773094113280,"available":772869111808,"internally_reserved":0,"allocated":3579904,"data_stored":2052472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000605"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:21.989537+0000","last_change":"2024-05-08T06:37:21.989537+0000","last_active":"2024-05-08T06:37:21.989537+0000","last_peered":"2024-05-08T06:37:21.989537+0000","last_clean":"2024-05-08T06:37:21.989537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:21.989537+0000","last_undegraded":"2024-05-08T06:37:21.989537+0000","last_fullsized":"2024-05-08T06:37:21.989537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T06:37:20.983346+0000","last_change":"2024-05-08T06:37:20.983346+0000","last_active":"2024-05-08T06:37:20.983346+0000","last_peered":"2024-05-08T06:37:20.983346+0000","last_clean":"2024-05-08T06:37:20.983346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T06:37:20.983346+0000","last_undegraded":"2024-05-08T06:37:20.983346+0000","last_fullsized":"2024-05-08T06:37:20.983346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-08T06:37:17.013395+0000","last_change":"2024-05-08T06:36:31.710287+0000","last_active":"2024-05-08T06:37:17.013395+0000","last_peered":"2024-05-08T06:37:17.013395+0000","last_clean":"2024-05-08T06:37:17.013395+0000","last_became_active":"2024-05-08T06:36:31.709270+0000","last_became_peered":"2024-05-08T06:36:31.709270+0000","last_unstale":"2024-05-08T06:37:17.013395+0000","last_undegraded":"2024-05-08T06:37:17.013395+0000","last_fullsized":"2024-05-08T06:37:17.013395+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_clean_scrub_stamp":"2024-05-08T06:36:30.122918+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:42.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":892928,"data_stored":699335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":299008,"data_stored":108967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":108967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":108967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":299008,"data_stored":108967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":299008,"data_stored":108967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":108967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":892928,"data_stored":699335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:37:26.016 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:37:26.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.154+0000 7f1944ad1640 1 Processor -- start 2024-05-08T06:37:26.154 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.154+0000 7f1944ad1640 1 -- start start 2024-05-08T06:37:26.155 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.155+0000 7f1944ad1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f19400b07b0 0x7f19400d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.155 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.155+0000 7f1944ad1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1940073140 0x7f1940160d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.155 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.155+0000 7f1944ad1640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1940147020 con 0x7f1940146c40 2024-05-08T06:37:26.155 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.155+0000 7f1944ad1640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f19400b5940 con 0x7f19400b07b0 2024-05-08T06:37:26.155 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.155+0000 7f1944ad1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f19400b5b90 con 0x7f1940073140 2024-05-08T06:37:26.155 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.156+0000 7f193dd9b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f19400b07b0 0x7f19400d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.155 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.156+0000 7f193dd9b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f19400b07b0 0x7f19400d0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53584/0 (socket says 172.21.15.163:53584) 2024-05-08T06:37:26.155 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.156+0000 7f193dd9b640 1 -- 172.21.15.163:0/1307635733 learned_addr learned my addr 172.21.15.163:0/1307635733 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:26.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.156+0000 7f193dd9b640 1 -- 172.21.15.163:0/1307635733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1940073140 msgr2=0x7f1940160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:26.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.156+0000 7f193dd9b640 1 --2- 172.21.15.163:0/1307635733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1940073140 0x7f1940160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.156+0000 7f193dd9b640 1 -- 172.21.15.163:0/1307635733 >> v1:172.21.15.192:6789/0 conn(0x7f1940146c40 legacy=0x7f1940072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:26.156 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.156+0000 7f193dd9b640 1 -- 172.21.15.163:0/1307635733 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19400b5e70 con 0x7f19400b07b0 2024-05-08T06:37:26.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.157+0000 7f193dd9b640 1 --2- 172.21.15.163:0/1307635733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19400b07b0 0x7f19400d0d50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f19400b5ac0 tx=0x7f192802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6f47aa5dd7a9b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:26.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.157+0000 7f193d59a640 1 -- 172.21.15.163:0/1307635733 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928037070 con 0x7f19400b07b0 2024-05-08T06:37:26.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.157+0000 7f193d59a640 1 -- 172.21.15.163:0/1307635733 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1928032440 con 0x7f19400b07b0 2024-05-08T06:37:26.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.157+0000 7f193d59a640 1 -- 172.21.15.163:0/1307635733 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928032810 con 0x7f19400b07b0 2024-05-08T06:37:26.157 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.157+0000 7f1944ad1640 1 -- 172.21.15.163:0/1307635733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19400b07b0 msgr2=0x7f19400d0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.158 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.157+0000 7f1944ad1640 1 --2- 172.21.15.163:0/1307635733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19400b07b0 0x7f19400d0d50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f19400b5ac0 tx=0x7f192802a1c0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.158 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.157+0000 7f1944ad1640 1 -- 172.21.15.163:0/1307635733 shutdown_connections 2024-05-08T06:37:26.158 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.157+0000 7f1944ad1640 1 --2- 172.21.15.163:0/1307635733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1940073140 0x7f1940160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.158 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.157+0000 7f1944ad1640 1 --2- 172.21.15.163:0/1307635733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19400b07b0 0x7f19400d0d50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.158 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.157+0000 7f1944ad1640 1 -- 172.21.15.163:0/1307635733 >> 172.21.15.163:0/1307635733 conn(0x7f194006c500 msgr2=0x7f194006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:26.158 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.157+0000 7f1944ad1640 1 -- 172.21.15.163:0/1307635733 shutdown_connections 2024-05-08T06:37:26.158 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.157+0000 7f1944ad1640 1 -- 172.21.15.163:0/1307635733 wait complete. 2024-05-08T06:37:26.160 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.159+0000 7f1944ad1640 1 Processor -- start 2024-05-08T06:37:26.160 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.159+0000 7f1944ad1640 1 -- start start 2024-05-08T06:37:26.160 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.159+0000 7f1944ad1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1940073140 0x7f19401f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.160 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.159+0000 7f1944ad1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1940146c40 0x7f194013fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.160 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.159+0000 7f1944ad1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f194014fdd0 0x7f194013c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.160 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.159+0000 7f1944ad1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f194015d7f0 con 0x7f1940146c40 2024-05-08T06:37:26.160 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.159+0000 7f1944ad1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f19400b6430 con 0x7f1940073140 2024-05-08T06:37:26.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.159+0000 7f1944ad1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f19400cdd20 con 0x7f194014fdd0 2024-05-08T06:37:26.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.159+0000 7f193dd9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1940146c40 0x7f194013fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f193dd9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1940146c40 0x7f194013fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53590/0 (socket says 172.21.15.163:53590) 2024-05-08T06:37:26.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f193dd9b640 1 -- 172.21.15.163:0/2032190519 learned_addr learned my addr 172.21.15.163:0/2032190519 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:26.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f193e59c640 1 --2- 172.21.15.163:0/2032190519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1940073140 0x7f19401f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f193ed9d640 1 --2- 172.21.15.163:0/2032190519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f194014fdd0 0x7f194013c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f193dd9b640 1 -- 172.21.15.163:0/2032190519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f194014fdd0 msgr2=0x7f194013c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f193dd9b640 1 --2- 172.21.15.163:0/2032190519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f194014fdd0 0x7f194013c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f193dd9b640 1 -- 172.21.15.163:0/2032190519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1940073140 msgr2=0x7f19401f65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f193dd9b640 1 --2- 172.21.15.163:0/2032190519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1940073140 0x7f19401f65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f193dd9b640 1 -- 172.21.15.163:0/2032190519 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1940150620 con 0x7f1940146c40 2024-05-08T06:37:26.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f193dd9b640 1 --2- 172.21.15.163:0/2032190519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1940146c40 0x7f194013fb70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f19280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:26.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f1936ffd640 1 -- 172.21.15.163:0/2032190519 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928041070 con 0x7f1940146c40 2024-05-08T06:37:26.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f1936ffd640 1 -- 172.21.15.163:0/2032190519 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f192803c870 con 0x7f1940146c40 2024-05-08T06:37:26.162 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f1944ad1640 1 -- 172.21.15.163:0/2032190519 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19401f6c80 con 0x7f1940146c40 2024-05-08T06:37:26.162 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f1936ffd640 1 -- 172.21.15.163:0/2032190519 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928037040 con 0x7f1940146c40 2024-05-08T06:37:26.162 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.160+0000 7f1944ad1640 1 -- 172.21.15.163:0/2032190519 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19401f7240 con 0x7f1940146c40 2024-05-08T06:37:26.162 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.161+0000 7f1944ad1640 1 -- 172.21.15.163:0/2032190519 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f194006a550 con 0x7f1940146c40 2024-05-08T06:37:26.163 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.162+0000 7f1936ffd640 1 -- 172.21.15.163:0/2032190519 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1928044070 con 0x7f1940146c40 2024-05-08T06:37:26.163 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.162+0000 7f1936ffd640 1 --2- 172.21.15.163:0/2032190519 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f19200762d0 0x7f1920078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.163 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.163+0000 7f193e59c640 1 --2- 172.21.15.163:0/2032190519 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f19200762d0 0x7f1920078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.163 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.163+0000 7f1936ffd640 1 -- 172.21.15.163:0/2032190519 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f19280b8050 con 0x7f1940146c40 2024-05-08T06:37:26.163 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.163+0000 7f193e59c640 1 --2- 172.21.15.163:0/2032190519 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f19200762d0 0x7f1920078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1938009f60 tx=0x7f19380094f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:26.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.167+0000 7f1936ffd640 1 -- 172.21.15.163:0/2032190519 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1928081a40 con 0x7f1940146c40 2024-05-08T06:37:26.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.346+0000 7f1944ad1640 1 -- 172.21.15.163:0/2032190519 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f194006cba0 con 0x7f19200762d0 2024-05-08T06:37:26.351 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.352+0000 7f1936ffd640 1 -- 172.21.15.163:0/2032190519 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251897 (secure 0 0 0) 0x7f194006cba0 con 0x7f19200762d0 2024-05-08T06:37:26.353 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:26.355 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2024-05-08T06:37:26.356 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.355+0000 7f1944ad1640 1 -- 172.21.15.163:0/2032190519 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f19200762d0 msgr2=0x7f1920078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.356 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.355+0000 7f1944ad1640 1 --2- 172.21.15.163:0/2032190519 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f19200762d0 0x7f1920078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1938009f60 tx=0x7f19380094f0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.356 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.355+0000 7f1944ad1640 1 -- 172.21.15.163:0/2032190519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1940146c40 msgr2=0x7f194013fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.356 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.355+0000 7f1944ad1640 1 --2- 172.21.15.163:0/2032190519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1940146c40 0x7f194013fb70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f19280046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.356 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.355+0000 7f1944ad1640 1 -- 172.21.15.163:0/2032190519 shutdown_connections 2024-05-08T06:37:26.356 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.355+0000 7f1944ad1640 1 --2- 172.21.15.163:0/2032190519 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f19200762d0 0x7f1920078790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.357 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.355+0000 7f1944ad1640 1 --2- 172.21.15.163:0/2032190519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f194014fdd0 0x7f194013c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.357 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.355+0000 7f1944ad1640 1 --2- 172.21.15.163:0/2032190519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1940146c40 0x7f194013fb70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.357 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.355+0000 7f1944ad1640 1 --2- 172.21.15.163:0/2032190519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1940073140 0x7f19401f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.357 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.355+0000 7f1944ad1640 1 -- 172.21.15.163:0/2032190519 >> 172.21.15.163:0/2032190519 conn(0x7f194006c500 msgr2=0x7f19400b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:26.357 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.355+0000 7f1944ad1640 1 -- 172.21.15.163:0/2032190519 shutdown_connections 2024-05-08T06:37:26.357 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.355+0000 7f1944ad1640 1 -- 172.21.15.163:0/2032190519 wait complete. 2024-05-08T06:37:26.365 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-05-08T06:37:26.106404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220056,"kb_used_data":3860,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754664,"statfs":{"total":773094113280,"available":772868775936,"internally_reserved":0,"allocated":3952640,"data_stored":2214464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.113839"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025117+0000","last_change":"2024-05-08T06:37:23.027600+0000","last_active":"2024-05-08T06:37:24.025117+0000","last_peered":"2024-05-08T06:37:24.025117+0000","last_clean":"2024-05-08T06:37:24.025117+0000","last_became_active":"2024-05-08T06:37:23.027276+0000","last_became_peered":"2024-05-08T06:37:23.027276+0000","last_unstale":"2024-05-08T06:37:24.025117+0000","last_undegraded":"2024-05-08T06:37:24.025117+0000","last_fullsized":"2024-05-08T06:37:24.025117+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:35.676798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.037512+0000","last_change":"2024-05-08T06:37:23.025427+0000","last_active":"2024-05-08T06:37:23.037512+0000","last_peered":"2024-05-08T06:37:23.037512+0000","last_clean":"2024-05-08T06:37:23.037512+0000","last_became_active":"2024-05-08T06:37:23.025164+0000","last_became_peered":"2024-05-08T06:37:23.025164+0000","last_unstale":"2024-05-08T06:37:23.037512+0000","last_undegraded":"2024-05-08T06:37:23.037512+0000","last_fullsized":"2024-05-08T06:37:23.037512+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:23.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.032857+0000","last_change":"2024-05-08T06:37:23.030355+0000","last_active":"2024-05-08T06:37:23.032857+0000","last_peered":"2024-05-08T06:37:23.032857+0000","last_clean":"2024-05-08T06:37:23.032857+0000","last_became_active":"2024-05-08T06:37:23.029913+0000","last_became_peered":"2024-05-08T06:37:23.029913+0000","last_unstale":"2024-05-08T06:37:23.032857+0000","last_undegraded":"2024-05-08T06:37:23.032857+0000","last_fullsized":"2024-05-08T06:37:23.032857+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:59.042090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.034968+0000","last_change":"2024-05-08T06:37:23.034487+0000","last_active":"2024-05-08T06:37:24.034968+0000","last_peered":"2024-05-08T06:37:24.034968+0000","last_clean":"2024-05-08T06:37:24.034968+0000","last_became_active":"2024-05-08T06:37:23.033975+0000","last_became_peered":"2024-05-08T06:37:23.033975+0000","last_unstale":"2024-05-08T06:37:24.034968+0000","last_undegraded":"2024-05-08T06:37:24.034968+0000","last_fullsized":"2024-05-08T06:37:24.034968+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:22.946893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.037755+0000","last_change":"2024-05-08T06:37:23.033451+0000","last_active":"2024-05-08T06:37:23.037755+0000","last_peered":"2024-05-08T06:37:23.037755+0000","last_clean":"2024-05-08T06:37:23.037755+0000","last_became_active":"2024-05-08T06:37:23.033168+0000","last_became_peered":"2024-05-08T06:37:23.033168+0000","last_unstale":"2024-05-08T06:37:23.037755+0000","last_undegraded":"2024-05-08T06:37:23.037755+0000","last_fullsized":"2024-05-08T06:37:23.037755+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:41.761170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.038227+0000","last_change":"2024-05-08T06:37:23.030910+0000","last_active":"2024-05-08T06:37:23.038227+0000","last_peered":"2024-05-08T06:37:23.038227+0000","last_clean":"2024-05-08T06:37:23.038227+0000","last_became_active":"2024-05-08T06:37:23.030110+0000","last_became_peered":"2024-05-08T06:37:23.030110+0000","last_unstale":"2024-05-08T06:37:23.038227+0000","last_undegraded":"2024-05-08T06:37:23.038227+0000","last_fullsized":"2024-05-08T06:37:23.038227+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:25.115721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.032302+0000","last_change":"2024-05-08T06:37:23.029564+0000","last_active":"2024-05-08T06:37:23.032302+0000","last_peered":"2024-05-08T06:37:23.032302+0000","last_clean":"2024-05-08T06:37:23.032302+0000","last_became_active":"2024-05-08T06:37:23.028614+0000","last_became_peered":"2024-05-08T06:37:23.028614+0000","last_unstale":"2024-05-08T06:37:23.032302+0000","last_undegraded":"2024-05-08T06:37:23.032302+0000","last_fullsized":"2024-05-08T06:37:23.032302+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:59.042090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.038408+0000","last_change":"2024-05-08T06:37:23.032789+0000","last_active":"2024-05-08T06:37:23.038408+0000","last_peered":"2024-05-08T06:37:23.038408+0000","last_clean":"2024-05-08T06:37:23.038408+0000","last_became_active":"2024-05-08T06:37:23.032548+0000","last_became_peered":"2024-05-08T06:37:23.032548+0000","last_unstale":"2024-05-08T06:37:23.038408+0000","last_undegraded":"2024-05-08T06:37:23.038408+0000","last_fullsized":"2024-05-08T06:37:23.038408+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:49.479743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.026159+0000","last_change":"2024-05-08T06:37:23.031376+0000","last_active":"2024-05-08T06:37:24.026159+0000","last_peered":"2024-05-08T06:37:24.026159+0000","last_clean":"2024-05-08T06:37:24.026159+0000","last_became_active":"2024-05-08T06:37:23.030746+0000","last_became_peered":"2024-05-08T06:37:23.030746+0000","last_unstale":"2024-05-08T06:37:24.026159+0000","last_undegraded":"2024-05-08T06:37:24.026159+0000","last_fullsized":"2024-05-08T06:37:24.026159+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:22.946893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.034994+0000","last_change":"2024-05-08T06:37:23.027803+0000","last_active":"2024-05-08T06:37:24.034994+0000","last_peered":"2024-05-08T06:37:24.034994+0000","last_clean":"2024-05-08T06:37:24.034994+0000","last_became_active":"2024-05-08T06:37:23.027553+0000","last_became_peered":"2024-05-08T06:37:23.027553+0000","last_unstale":"2024-05-08T06:37:24.034994+0000","last_undegraded":"2024-05-08T06:37:24.034994+0000","last_fullsized":"2024-05-08T06:37:24.034994+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:25.866311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.038635+0000","last_change":"2024-05-08T06:37:23.032448+0000","last_active":"2024-05-08T06:37:23.038635+0000","last_peered":"2024-05-08T06:37:23.038635+0000","last_clean":"2024-05-08T06:37:23.038635+0000","last_became_active":"2024-05-08T06:37:23.032144+0000","last_became_peered":"2024-05-08T06:37:23.032144+0000","last_unstale":"2024-05-08T06:37:23.038635+0000","last_undegraded":"2024-05-08T06:37:23.038635+0000","last_fullsized":"2024-05-08T06:37:23.038635+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:49.411213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.041794+0000","last_change":"2024-05-08T06:37:23.040119+0000","last_active":"2024-05-08T06:37:23.041794+0000","last_peered":"2024-05-08T06:37:23.041794+0000","last_clean":"2024-05-08T06:37:23.041794+0000","last_became_active":"2024-05-08T06:37:23.039713+0000","last_became_peered":"2024-05-08T06:37:23.039713+0000","last_unstale":"2024-05-08T06:37:23.041794+0000","last_undegraded":"2024-05-08T06:37:23.041794+0000","last_fullsized":"2024-05-08T06:37:23.041794+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:23.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.029462+0000","last_change":"2024-05-08T06:37:23.028444+0000","last_active":"2024-05-08T06:37:23.029462+0000","last_peered":"2024-05-08T06:37:23.029462+0000","last_clean":"2024-05-08T06:37:23.029462+0000","last_became_active":"2024-05-08T06:37:23.028147+0000","last_became_peered":"2024-05-08T06:37:23.028147+0000","last_unstale":"2024-05-08T06:37:23.029462+0000","last_undegraded":"2024-05-08T06:37:23.029462+0000","last_fullsized":"2024-05-08T06:37:23.029462+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:43.605752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.038203+0000","last_change":"2024-05-08T06:37:23.030566+0000","last_active":"2024-05-08T06:37:23.038203+0000","last_peered":"2024-05-08T06:37:23.038203+0000","last_clean":"2024-05-08T06:37:23.038203+0000","last_became_active":"2024-05-08T06:37:23.030304+0000","last_became_peered":"2024-05-08T06:37:23.030304+0000","last_unstale":"2024-05-08T06:37:23.038203+0000","last_undegraded":"2024-05-08T06:37:23.038203+0000","last_fullsized":"2024-05-08T06:37:23.038203+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:22.946893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.035447+0000","last_change":"2024-05-08T06:37:23.030952+0000","last_active":"2024-05-08T06:37:23.035447+0000","last_peered":"2024-05-08T06:37:23.035447+0000","last_clean":"2024-05-08T06:37:23.035447+0000","last_became_active":"2024-05-08T06:37:23.030577+0000","last_became_peered":"2024-05-08T06:37:23.030577+0000","last_unstale":"2024-05-08T06:37:23.035447+0000","last_undegraded":"2024-05-08T06:37:23.035447+0000","last_fullsized":"2024-05-08T06:37:23.035447+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:34.655747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.041712+0000","last_change":"2024-05-08T06:37:23.040069+0000","last_active":"2024-05-08T06:37:23.041712+0000","last_peered":"2024-05-08T06:37:23.041712+0000","last_clean":"2024-05-08T06:37:23.041712+0000","last_became_active":"2024-05-08T06:37:23.039686+0000","last_became_peered":"2024-05-08T06:37:23.039686+0000","last_unstale":"2024-05-08T06:37:23.041712+0000","last_undegraded":"2024-05-08T06:37:23.041712+0000","last_fullsized":"2024-05-08T06:37:23.041712+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:25.866311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.035764+0000","last_change":"2024-05-08T06:37:23.030775+0000","last_active":"2024-05-08T06:37:23.035764+0000","last_peered":"2024-05-08T06:37:23.035764+0000","last_clean":"2024-05-08T06:37:23.035764+0000","last_became_active":"2024-05-08T06:37:23.030524+0000","last_became_peered":"2024-05-08T06:37:23.030524+0000","last_unstale":"2024-05-08T06:37:23.035764+0000","last_undegraded":"2024-05-08T06:37:23.035764+0000","last_fullsized":"2024-05-08T06:37:23.035764+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:34.655747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.029852+0000","last_change":"2024-05-08T06:37:23.023504+0000","last_active":"2024-05-08T06:37:23.029852+0000","last_peered":"2024-05-08T06:37:23.029852+0000","last_clean":"2024-05-08T06:37:23.029852+0000","last_became_active":"2024-05-08T06:37:23.023263+0000","last_became_peered":"2024-05-08T06:37:23.023263+0000","last_unstale":"2024-05-08T06:37:23.029852+0000","last_undegraded":"2024-05-08T06:37:23.029852+0000","last_fullsized":"2024-05-08T06:37:23.029852+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:01.330678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.029219+0000","last_change":"2024-05-08T06:37:23.022359+0000","last_active":"2024-05-08T06:37:23.029219+0000","last_peered":"2024-05-08T06:37:23.029219+0000","last_clean":"2024-05-08T06:37:23.029219+0000","last_became_active":"2024-05-08T06:37:23.022060+0000","last_became_peered":"2024-05-08T06:37:23.022060+0000","last_unstale":"2024-05-08T06:37:23.029219+0000","last_undegraded":"2024-05-08T06:37:23.029219+0000","last_fullsized":"2024-05-08T06:37:23.029219+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:34.655747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.033744+0000","last_change":"2024-05-08T06:37:23.032169+0000","last_active":"2024-05-08T06:37:23.033744+0000","last_peered":"2024-05-08T06:37:23.033744+0000","last_clean":"2024-05-08T06:37:23.033744+0000","last_became_active":"2024-05-08T06:37:23.031893+0000","last_became_peered":"2024-05-08T06:37:23.031893+0000","last_unstale":"2024-05-08T06:37:23.033744+0000","last_undegraded":"2024-05-08T06:37:23.033744+0000","last_fullsized":"2024-05-08T06:37:23.033744+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:25:23.810377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025474+0000","last_change":"2024-05-08T06:37:23.026119+0000","last_active":"2024-05-08T06:37:24.025474+0000","last_peered":"2024-05-08T06:37:24.025474+0000","last_clean":"2024-05-08T06:37:24.025474+0000","last_became_active":"2024-05-08T06:37:23.025633+0000","last_became_peered":"2024-05-08T06:37:23.025633+0000","last_unstale":"2024-05-08T06:37:24.025474+0000","last_undegraded":"2024-05-08T06:37:24.025474+0000","last_fullsized":"2024-05-08T06:37:24.025474+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:49.411213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025574+0000","last_change":"2024-05-08T06:37:23.025969+0000","last_active":"2024-05-08T06:37:24.025574+0000","last_peered":"2024-05-08T06:37:24.025574+0000","last_clean":"2024-05-08T06:37:24.025574+0000","last_became_active":"2024-05-08T06:37:23.025146+0000","last_became_peered":"2024-05-08T06:37:23.025146+0000","last_unstale":"2024-05-08T06:37:24.025574+0000","last_undegraded":"2024-05-08T06:37:24.025574+0000","last_fullsized":"2024-05-08T06:37:24.025574+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:49.479743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.024032+0000","last_change":"2024-05-08T06:37:23.034613+0000","last_active":"2024-05-08T06:37:24.024032+0000","last_peered":"2024-05-08T06:37:24.024032+0000","last_clean":"2024-05-08T06:37:24.024032+0000","last_became_active":"2024-05-08T06:37:23.034319+0000","last_became_peered":"2024-05-08T06:37:23.034319+0000","last_unstale":"2024-05-08T06:37:24.024032+0000","last_undegraded":"2024-05-08T06:37:24.024032+0000","last_fullsized":"2024-05-08T06:37:24.024032+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:34.655747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.032545+0000","last_change":"2024-05-08T06:37:23.030317+0000","last_active":"2024-05-08T06:37:23.032545+0000","last_peered":"2024-05-08T06:37:23.032545+0000","last_clean":"2024-05-08T06:37:23.032545+0000","last_became_active":"2024-05-08T06:37:23.030043+0000","last_became_peered":"2024-05-08T06:37:23.030043+0000","last_unstale":"2024-05-08T06:37:23.032545+0000","last_undegraded":"2024-05-08T06:37:23.032545+0000","last_fullsized":"2024-05-08T06:37:23.032545+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:35.676798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.035098+0000","last_change":"2024-05-08T06:37:23.035436+0000","last_active":"2024-05-08T06:37:24.035098+0000","last_peered":"2024-05-08T06:37:24.035098+0000","last_clean":"2024-05-08T06:37:24.035098+0000","last_became_active":"2024-05-08T06:37:23.035039+0000","last_became_peered":"2024-05-08T06:37:23.035039+0000","last_unstale":"2024-05-08T06:37:24.035098+0000","last_undegraded":"2024-05-08T06:37:24.035098+0000","last_fullsized":"2024-05-08T06:37:24.035098+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:51:47.146271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.037638+0000","last_change":"2024-05-08T06:37:23.030688+0000","last_active":"2024-05-08T06:37:23.037638+0000","last_peered":"2024-05-08T06:37:23.037638+0000","last_clean":"2024-05-08T06:37:23.037638+0000","last_became_active":"2024-05-08T06:37:23.030410+0000","last_became_peered":"2024-05-08T06:37:23.030410+0000","last_unstale":"2024-05-08T06:37:23.037638+0000","last_undegraded":"2024-05-08T06:37:23.037638+0000","last_fullsized":"2024-05-08T06:37:23.037638+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:25.866311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.035066+0000","last_change":"2024-05-08T06:37:23.034971+0000","last_active":"2024-05-08T06:37:24.035066+0000","last_peered":"2024-05-08T06:37:24.035066+0000","last_clean":"2024-05-08T06:37:24.035066+0000","last_became_active":"2024-05-08T06:37:23.034701+0000","last_became_peered":"2024-05-08T06:37:23.034701+0000","last_unstale":"2024-05-08T06:37:24.035066+0000","last_undegraded":"2024-05-08T06:37:24.035066+0000","last_fullsized":"2024-05-08T06:37:24.035066+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:45.637944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.032523+0000","last_change":"2024-05-08T06:37:23.029392+0000","last_active":"2024-05-08T06:37:23.032523+0000","last_peered":"2024-05-08T06:37:23.032523+0000","last_clean":"2024-05-08T06:37:23.032523+0000","last_became_active":"2024-05-08T06:37:23.029105+0000","last_became_peered":"2024-05-08T06:37:23.029105+0000","last_unstale":"2024-05-08T06:37:23.032523+0000","last_undegraded":"2024-05-08T06:37:23.032523+0000","last_fullsized":"2024-05-08T06:37:23.032523+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:43:47.264013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025657+0000","last_change":"2024-05-08T06:37:23.026424+0000","last_active":"2024-05-08T06:37:24.025657+0000","last_peered":"2024-05-08T06:37:24.025657+0000","last_clean":"2024-05-08T06:37:24.025657+0000","last_became_active":"2024-05-08T06:37:23.026070+0000","last_became_peered":"2024-05-08T06:37:23.026070+0000","last_unstale":"2024-05-08T06:37:24.025657+0000","last_undegraded":"2024-05-08T06:37:24.025657+0000","last_fullsized":"2024-05-08T06:37:24.025657+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:01.330678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.030073+0000","last_change":"2024-05-08T06:37:23.023756+0000","last_active":"2024-05-08T06:37:23.030073+0000","last_peered":"2024-05-08T06:37:23.030073+0000","last_clean":"2024-05-08T06:37:23.030073+0000","last_became_active":"2024-05-08T06:37:23.023281+0000","last_became_peered":"2024-05-08T06:37:23.023281+0000","last_unstale":"2024-05-08T06:37:23.030073+0000","last_undegraded":"2024-05-08T06:37:23.030073+0000","last_fullsized":"2024-05-08T06:37:23.030073+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:43:47.264013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025044+0000","last_change":"2024-05-08T06:37:23.029379+0000","last_active":"2024-05-08T06:37:24.025044+0000","last_peered":"2024-05-08T06:37:24.025044+0000","last_clean":"2024-05-08T06:37:24.025044+0000","last_became_active":"2024-05-08T06:37:23.029106+0000","last_became_peered":"2024-05-08T06:37:23.029106+0000","last_unstale":"2024-05-08T06:37:24.025044+0000","last_undegraded":"2024-05-08T06:37:24.025044+0000","last_fullsized":"2024-05-08T06:37:24.025044+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:17.531271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.039087+0000","last_change":"2024-05-08T06:37:23.033628+0000","last_active":"2024-05-08T06:37:23.039087+0000","last_peered":"2024-05-08T06:37:23.039087+0000","last_clean":"2024-05-08T06:37:23.039087+0000","last_became_active":"2024-05-08T06:37:23.033377+0000","last_became_peered":"2024-05-08T06:37:23.033377+0000","last_unstale":"2024-05-08T06:37:23.039087+0000","last_undegraded":"2024-05-08T06:37:23.039087+0000","last_fullsized":"2024-05-08T06:37:23.039087+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:23.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.024133+0000","last_change":"2024-05-08T06:37:23.026057+0000","last_active":"2024-05-08T06:37:24.024133+0000","last_peered":"2024-05-08T06:37:24.024133+0000","last_clean":"2024-05-08T06:37:24.024133+0000","last_became_active":"2024-05-08T06:37:23.025717+0000","last_became_peered":"2024-05-08T06:37:23.025717+0000","last_unstale":"2024-05-08T06:37:24.024133+0000","last_undegraded":"2024-05-08T06:37:24.024133+0000","last_fullsized":"2024-05-08T06:37:24.024133+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:49.411213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025220+0000","last_change":"2024-05-08T06:37:22.005562+0000","last_active":"2024-05-08T06:37:24.025220+0000","last_peered":"2024-05-08T06:37:24.025220+0000","last_clean":"2024-05-08T06:37:24.025220+0000","last_became_active":"2024-05-08T06:37:22.005284+0000","last_became_peered":"2024-05-08T06:37:22.005284+0000","last_unstale":"2024-05-08T06:37:24.025220+0000","last_undegraded":"2024-05-08T06:37:24.025220+0000","last_fullsized":"2024-05-08T06:37:24.025220+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:33:43.783546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025171+0000","last_change":"2024-05-08T06:37:23.025014+0000","last_active":"2024-05-08T06:37:24.025171+0000","last_peered":"2024-05-08T06:37:24.025171+0000","last_clean":"2024-05-08T06:37:24.025171+0000","last_became_active":"2024-05-08T06:37:23.024736+0000","last_became_peered":"2024-05-08T06:37:23.024736+0000","last_unstale":"2024-05-08T06:37:24.025171+0000","last_undegraded":"2024-05-08T06:37:24.025171+0000","last_fullsized":"2024-05-08T06:37:24.025171+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:59.042090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.024150+0000","last_change":"2024-05-08T06:37:22.009932+0000","last_active":"2024-05-08T06:37:24.024150+0000","last_peered":"2024-05-08T06:37:24.024150+0000","last_clean":"2024-05-08T06:37:24.024150+0000","last_became_active":"2024-05-08T06:37:22.009624+0000","last_became_peered":"2024-05-08T06:37:22.009624+0000","last_unstale":"2024-05-08T06:37:24.024150+0000","last_undegraded":"2024-05-08T06:37:24.024150+0000","last_fullsized":"2024-05-08T06:37:24.024150+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:29:02.885057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025843+0000","last_change":"2024-05-08T06:37:23.026650+0000","last_active":"2024-05-08T06:37:24.025843+0000","last_peered":"2024-05-08T06:37:24.025843+0000","last_clean":"2024-05-08T06:37:24.025843+0000","last_became_active":"2024-05-08T06:37:23.025913+0000","last_became_peered":"2024-05-08T06:37:23.025913+0000","last_unstale":"2024-05-08T06:37:24.025843+0000","last_undegraded":"2024-05-08T06:37:24.025843+0000","last_fullsized":"2024-05-08T06:37:24.025843+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:16.749074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.045145+0000","last_change":"2024-05-08T06:37:22.014889+0000","last_active":"2024-05-08T06:37:23.045145+0000","last_peered":"2024-05-08T06:37:23.045145+0000","last_clean":"2024-05-08T06:37:23.045145+0000","last_became_active":"2024-05-08T06:37:22.014642+0000","last_became_peered":"2024-05-08T06:37:22.014642+0000","last_unstale":"2024-05-08T06:37:23.045145+0000","last_undegraded":"2024-05-08T06:37:23.045145+0000","last_fullsized":"2024-05-08T06:37:23.045145+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:26:06.644693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.035053+0000","last_change":"2024-05-08T06:37:23.032386+0000","last_active":"2024-05-08T06:37:24.035053+0000","last_peered":"2024-05-08T06:37:24.035053+0000","last_clean":"2024-05-08T06:37:24.035053+0000","last_became_active":"2024-05-08T06:37:23.032105+0000","last_became_peered":"2024-05-08T06:37:23.032105+0000","last_unstale":"2024-05-08T06:37:24.035053+0000","last_undegraded":"2024-05-08T06:37:24.035053+0000","last_fullsized":"2024-05-08T06:37:24.035053+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:01:57.210011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.041314+0000","last_change":"2024-05-08T06:37:22.011516+0000","last_active":"2024-05-08T06:37:23.041314+0000","last_peered":"2024-05-08T06:37:23.041314+0000","last_clean":"2024-05-08T06:37:23.041314+0000","last_became_active":"2024-05-08T06:37:22.011193+0000","last_became_peered":"2024-05-08T06:37:22.011193+0000","last_unstale":"2024-05-08T06:37:23.041314+0000","last_undegraded":"2024-05-08T06:37:23.041314+0000","last_fullsized":"2024-05-08T06:37:23.041314+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:47.658277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.039008+0000","last_change":"2024-05-08T06:37:23.033036+0000","last_active":"2024-05-08T06:37:23.039008+0000","last_peered":"2024-05-08T06:37:23.039008+0000","last_clean":"2024-05-08T06:37:23.039008+0000","last_became_active":"2024-05-08T06:37:23.032713+0000","last_became_peered":"2024-05-08T06:37:23.032713+0000","last_unstale":"2024-05-08T06:37:23.039008+0000","last_undegraded":"2024-05-08T06:37:23.039008+0000","last_fullsized":"2024-05-08T06:37:23.039008+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:16.749074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025682+0000","last_change":"2024-05-08T06:37:22.007046+0000","last_active":"2024-05-08T06:37:24.025682+0000","last_peered":"2024-05-08T06:37:24.025682+0000","last_clean":"2024-05-08T06:37:24.025682+0000","last_became_active":"2024-05-08T06:37:22.006627+0000","last_became_peered":"2024-05-08T06:37:22.006627+0000","last_unstale":"2024-05-08T06:37:24.025682+0000","last_undegraded":"2024-05-08T06:37:24.025682+0000","last_fullsized":"2024-05-08T06:37:24.025682+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:41:40.880339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.035422+0000","last_change":"2024-05-08T06:37:23.031131+0000","last_active":"2024-05-08T06:37:23.035422+0000","last_peered":"2024-05-08T06:37:23.035422+0000","last_clean":"2024-05-08T06:37:23.035422+0000","last_became_active":"2024-05-08T06:37:23.030866+0000","last_became_peered":"2024-05-08T06:37:23.030866+0000","last_unstale":"2024-05-08T06:37:23.035422+0000","last_undegraded":"2024-05-08T06:37:23.035422+0000","last_fullsized":"2024-05-08T06:37:23.035422+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:49.411213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025558+0000","last_change":"2024-05-08T06:37:22.007379+0000","last_active":"2024-05-08T06:37:24.025558+0000","last_peered":"2024-05-08T06:37:24.025558+0000","last_clean":"2024-05-08T06:37:24.025558+0000","last_became_active":"2024-05-08T06:37:22.006988+0000","last_became_peered":"2024-05-08T06:37:22.006988+0000","last_unstale":"2024-05-08T06:37:24.025558+0000","last_undegraded":"2024-05-08T06:37:24.025558+0000","last_fullsized":"2024-05-08T06:37:24.025558+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:58.710469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.024738+0000","last_change":"2024-05-08T06:37:23.020918+0000","last_active":"2024-05-08T06:37:24.024738+0000","last_peered":"2024-05-08T06:37:24.024738+0000","last_clean":"2024-05-08T06:37:24.024738+0000","last_became_active":"2024-05-08T06:37:23.020600+0000","last_became_peered":"2024-05-08T06:37:23.020600+0000","last_unstale":"2024-05-08T06:37:24.024738+0000","last_undegraded":"2024-05-08T06:37:24.024738+0000","last_fullsized":"2024-05-08T06:37:24.024738+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:34.655747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.014961+0000","last_change":"2024-05-08T06:37:22.004489+0000","last_active":"2024-05-08T06:37:23.014961+0000","last_peered":"2024-05-08T06:37:23.014961+0000","last_clean":"2024-05-08T06:37:23.014961+0000","last_became_active":"2024-05-08T06:37:22.004259+0000","last_became_peered":"2024-05-08T06:37:22.004259+0000","last_unstale":"2024-05-08T06:37:23.014961+0000","last_undegraded":"2024-05-08T06:37:23.014961+0000","last_fullsized":"2024-05-08T06:37:23.014961+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:36:35.031102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.034751+0000","last_change":"2024-05-08T06:37:23.027159+0000","last_active":"2024-05-08T06:37:24.034751+0000","last_peered":"2024-05-08T06:37:24.034751+0000","last_clean":"2024-05-08T06:37:24.034751+0000","last_became_active":"2024-05-08T06:37:23.026869+0000","last_became_peered":"2024-05-08T06:37:23.026869+0000","last_unstale":"2024-05-08T06:37:24.034751+0000","last_undegraded":"2024-05-08T06:37:24.034751+0000","last_fullsized":"2024-05-08T06:37:24.034751+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:16.749074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.024593+0000","last_change":"2024-05-08T06:37:22.003564+0000","last_active":"2024-05-08T06:37:24.024593+0000","last_peered":"2024-05-08T06:37:24.024593+0000","last_clean":"2024-05-08T06:37:24.024593+0000","last_became_active":"2024-05-08T06:37:22.003269+0000","last_became_peered":"2024-05-08T06:37:22.003269+0000","last_unstale":"2024-05-08T06:37:24.024593+0000","last_undegraded":"2024-05-08T06:37:24.024593+0000","last_fullsized":"2024-05-08T06:37:24.024593+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:15.248123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.041800+0000","last_change":"2024-05-08T06:37:23.040263+0000","last_active":"2024-05-08T06:37:23.041800+0000","last_peered":"2024-05-08T06:37:23.041800+0000","last_clean":"2024-05-08T06:37:23.041800+0000","last_became_active":"2024-05-08T06:37:23.039738+0000","last_became_peered":"2024-05-08T06:37:23.039738+0000","last_unstale":"2024-05-08T06:37:23.041800+0000","last_undegraded":"2024-05-08T06:37:23.041800+0000","last_fullsized":"2024-05-08T06:37:23.041800+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:22.946893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.015105+0000","last_change":"2024-05-08T06:37:22.011743+0000","last_active":"2024-05-08T06:37:23.015105+0000","last_peered":"2024-05-08T06:37:23.015105+0000","last_clean":"2024-05-08T06:37:23.015105+0000","last_became_active":"2024-05-08T06:37:22.011459+0000","last_became_peered":"2024-05-08T06:37:22.011459+0000","last_unstale":"2024-05-08T06:37:23.015105+0000","last_undegraded":"2024-05-08T06:37:23.015105+0000","last_fullsized":"2024-05-08T06:37:23.015105+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:16.525080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.028879+0000","last_change":"2024-05-08T06:37:23.022409+0000","last_active":"2024-05-08T06:37:23.028879+0000","last_peered":"2024-05-08T06:37:23.028879+0000","last_clean":"2024-05-08T06:37:23.028879+0000","last_became_active":"2024-05-08T06:37:23.022122+0000","last_became_peered":"2024-05-08T06:37:23.022122+0000","last_unstale":"2024-05-08T06:37:23.028879+0000","last_undegraded":"2024-05-08T06:37:23.028879+0000","last_fullsized":"2024-05-08T06:37:23.028879+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:49.411213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.024214+0000","last_change":"2024-05-08T06:37:22.006278+0000","last_active":"2024-05-08T06:37:24.024214+0000","last_peered":"2024-05-08T06:37:24.024214+0000","last_clean":"2024-05-08T06:37:24.024214+0000","last_became_active":"2024-05-08T06:37:22.005898+0000","last_became_peered":"2024-05-08T06:37:22.005898+0000","last_unstale":"2024-05-08T06:37:24.024214+0000","last_undegraded":"2024-05-08T06:37:24.024214+0000","last_fullsized":"2024-05-08T06:37:24.024214+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:36:35.031102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.038400+0000","last_change":"2024-05-08T06:37:23.031513+0000","last_active":"2024-05-08T06:37:23.038400+0000","last_peered":"2024-05-08T06:37:23.038400+0000","last_clean":"2024-05-08T06:37:23.038400+0000","last_became_active":"2024-05-08T06:37:23.031213+0000","last_became_peered":"2024-05-08T06:37:23.031213+0000","last_unstale":"2024-05-08T06:37:23.038400+0000","last_undegraded":"2024-05-08T06:37:23.038400+0000","last_fullsized":"2024-05-08T06:37:23.038400+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:01:57.210011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.041300+0000","last_change":"2024-05-08T06:37:22.011714+0000","last_active":"2024-05-08T06:37:23.041300+0000","last_peered":"2024-05-08T06:37:23.041300+0000","last_clean":"2024-05-08T06:37:23.041300+0000","last_became_active":"2024-05-08T06:37:22.011256+0000","last_became_peered":"2024-05-08T06:37:22.011256+0000","last_unstale":"2024-05-08T06:37:23.041300+0000","last_undegraded":"2024-05-08T06:37:23.041300+0000","last_fullsized":"2024-05-08T06:37:23.041300+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:26:06.644693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.035494+0000","last_change":"2024-05-08T06:37:23.031912+0000","last_active":"2024-05-08T06:37:23.035494+0000","last_peered":"2024-05-08T06:37:23.035494+0000","last_clean":"2024-05-08T06:37:23.035494+0000","last_became_active":"2024-05-08T06:37:23.031640+0000","last_became_peered":"2024-05-08T06:37:23.031640+0000","last_unstale":"2024-05-08T06:37:23.035494+0000","last_undegraded":"2024-05-08T06:37:23.035494+0000","last_fullsized":"2024-05-08T06:37:23.035494+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:49.479743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.047597+0000","last_change":"2024-05-08T06:37:22.014145+0000","last_active":"2024-05-08T06:37:23.047597+0000","last_peered":"2024-05-08T06:37:23.047597+0000","last_clean":"2024-05-08T06:37:23.047597+0000","last_became_active":"2024-05-08T06:37:22.013871+0000","last_became_peered":"2024-05-08T06:37:22.013871+0000","last_unstale":"2024-05-08T06:37:23.047597+0000","last_undegraded":"2024-05-08T06:37:23.047597+0000","last_fullsized":"2024-05-08T06:37:23.047597+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:43:46.257822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.034799+0000","last_change":"2024-05-08T06:37:23.024642+0000","last_active":"2024-05-08T06:37:24.034799+0000","last_peered":"2024-05-08T06:37:24.034799+0000","last_clean":"2024-05-08T06:37:24.034799+0000","last_became_active":"2024-05-08T06:37:23.024355+0000","last_became_peered":"2024-05-08T06:37:23.024355+0000","last_unstale":"2024-05-08T06:37:24.034799+0000","last_undegraded":"2024-05-08T06:37:24.034799+0000","last_fullsized":"2024-05-08T06:37:24.034799+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:23.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025766+0000","last_change":"2024-05-08T06:37:22.003394+0000","last_active":"2024-05-08T06:37:24.025766+0000","last_peered":"2024-05-08T06:37:24.025766+0000","last_clean":"2024-05-08T06:37:24.025766+0000","last_became_active":"2024-05-08T06:37:22.002818+0000","last_became_peered":"2024-05-08T06:37:22.002818+0000","last_unstale":"2024-05-08T06:37:24.025766+0000","last_undegraded":"2024-05-08T06:37:24.025766+0000","last_fullsized":"2024-05-08T06:37:24.025766+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:47.975288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.037727+0000","last_change":"2024-05-08T06:37:23.024861+0000","last_active":"2024-05-08T06:37:23.037727+0000","last_peered":"2024-05-08T06:37:23.037727+0000","last_clean":"2024-05-08T06:37:23.037727+0000","last_became_active":"2024-05-08T06:37:23.024572+0000","last_became_peered":"2024-05-08T06:37:23.024572+0000","last_unstale":"2024-05-08T06:37:23.037727+0000","last_undegraded":"2024-05-08T06:37:23.037727+0000","last_fullsized":"2024-05-08T06:37:23.037727+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:16.749074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.034713+0000","last_change":"2024-05-08T06:37:22.013595+0000","last_active":"2024-05-08T06:37:24.034713+0000","last_peered":"2024-05-08T06:37:24.034713+0000","last_clean":"2024-05-08T06:37:24.034713+0000","last_became_active":"2024-05-08T06:37:22.013318+0000","last_became_peered":"2024-05-08T06:37:22.013318+0000","last_unstale":"2024-05-08T06:37:24.034713+0000","last_undegraded":"2024-05-08T06:37:24.034713+0000","last_fullsized":"2024-05-08T06:37:24.034713+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:13.378762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.039120+0000","last_change":"2024-05-08T06:37:23.034398+0000","last_active":"2024-05-08T06:37:23.039120+0000","last_peered":"2024-05-08T06:37:23.039120+0000","last_clean":"2024-05-08T06:37:23.039120+0000","last_became_active":"2024-05-08T06:37:23.034133+0000","last_became_peered":"2024-05-08T06:37:23.034133+0000","last_unstale":"2024-05-08T06:37:23.039120+0000","last_undegraded":"2024-05-08T06:37:23.039120+0000","last_fullsized":"2024-05-08T06:37:23.039120+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:22.946893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.015104+0000","last_change":"2024-05-08T06:37:22.011690+0000","last_active":"2024-05-08T06:37:23.015104+0000","last_peered":"2024-05-08T06:37:23.015104+0000","last_clean":"2024-05-08T06:37:23.015104+0000","last_became_active":"2024-05-08T06:37:22.011414+0000","last_became_peered":"2024-05-08T06:37:22.011414+0000","last_unstale":"2024-05-08T06:37:23.015104+0000","last_undegraded":"2024-05-08T06:37:23.015104+0000","last_fullsized":"2024-05-08T06:37:23.015104+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:13.378762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025054+0000","last_change":"2024-05-08T06:37:23.024488+0000","last_active":"2024-05-08T06:37:24.025054+0000","last_peered":"2024-05-08T06:37:24.025054+0000","last_clean":"2024-05-08T06:37:24.025054+0000","last_became_active":"2024-05-08T06:37:23.024146+0000","last_became_peered":"2024-05-08T06:37:23.024146+0000","last_unstale":"2024-05-08T06:37:24.025054+0000","last_undegraded":"2024-05-08T06:37:24.025054+0000","last_fullsized":"2024-05-08T06:37:24.025054+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:26:07.650884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.045638+0000","last_change":"2024-05-08T06:37:22.014983+0000","last_active":"2024-05-08T06:37:23.045638+0000","last_peered":"2024-05-08T06:37:23.045638+0000","last_clean":"2024-05-08T06:37:23.045638+0000","last_became_active":"2024-05-08T06:37:22.014669+0000","last_became_peered":"2024-05-08T06:37:22.014669+0000","last_unstale":"2024-05-08T06:37:23.045638+0000","last_undegraded":"2024-05-08T06:37:23.045638+0000","last_fullsized":"2024-05-08T06:37:23.045638+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:34.670607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.026082+0000","last_change":"2024-05-08T06:37:23.027187+0000","last_active":"2024-05-08T06:37:24.026082+0000","last_peered":"2024-05-08T06:37:24.026082+0000","last_clean":"2024-05-08T06:37:24.026082+0000","last_became_active":"2024-05-08T06:37:23.026837+0000","last_became_peered":"2024-05-08T06:37:23.026837+0000","last_unstale":"2024-05-08T06:37:24.026082+0000","last_undegraded":"2024-05-08T06:37:24.026082+0000","last_fullsized":"2024-05-08T06:37:24.026082+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:23.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.035189+0000","last_change":"2024-05-08T06:37:22.011164+0000","last_active":"2024-05-08T06:37:24.035189+0000","last_peered":"2024-05-08T06:37:24.035189+0000","last_clean":"2024-05-08T06:37:24.035189+0000","last_became_active":"2024-05-08T06:37:22.010894+0000","last_became_peered":"2024-05-08T06:37:22.010894+0000","last_unstale":"2024-05-08T06:37:24.035189+0000","last_undegraded":"2024-05-08T06:37:24.035189+0000","last_fullsized":"2024-05-08T06:37:24.035189+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:52:39.204187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.039203+0000","last_change":"2024-05-08T06:37:23.032699+0000","last_active":"2024-05-08T06:37:23.039203+0000","last_peered":"2024-05-08T06:37:23.039203+0000","last_clean":"2024-05-08T06:37:23.039203+0000","last_became_active":"2024-05-08T06:37:23.032408+0000","last_became_peered":"2024-05-08T06:37:23.032408+0000","last_unstale":"2024-05-08T06:37:23.039203+0000","last_undegraded":"2024-05-08T06:37:23.039203+0000","last_fullsized":"2024-05-08T06:37:23.039203+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:01.330678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.026116+0000","last_change":"2024-05-08T06:37:22.007558+0000","last_active":"2024-05-08T06:37:24.026116+0000","last_peered":"2024-05-08T06:37:24.026116+0000","last_clean":"2024-05-08T06:37:24.026116+0000","last_became_active":"2024-05-08T06:37:22.006520+0000","last_became_peered":"2024-05-08T06:37:22.006520+0000","last_unstale":"2024-05-08T06:37:24.026116+0000","last_undegraded":"2024-05-08T06:37:24.026116+0000","last_fullsized":"2024-05-08T06:37:24.026116+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:37:13.012033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.035043+0000","last_change":"2024-05-08T06:37:23.034255+0000","last_active":"2024-05-08T06:37:24.035043+0000","last_peered":"2024-05-08T06:37:24.035043+0000","last_clean":"2024-05-08T06:37:24.035043+0000","last_became_active":"2024-05-08T06:37:23.033236+0000","last_became_peered":"2024-05-08T06:37:23.033236+0000","last_unstale":"2024-05-08T06:37:24.035043+0000","last_undegraded":"2024-05-08T06:37:24.035043+0000","last_fullsized":"2024-05-08T06:37:24.035043+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:25.115721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.026109+0000","last_change":"2024-05-08T06:37:22.006566+0000","last_active":"2024-05-08T06:37:24.026109+0000","last_peered":"2024-05-08T06:37:24.026109+0000","last_clean":"2024-05-08T06:37:24.026109+0000","last_became_active":"2024-05-08T06:37:22.006266+0000","last_became_peered":"2024-05-08T06:37:22.006266+0000","last_unstale":"2024-05-08T06:37:24.026109+0000","last_undegraded":"2024-05-08T06:37:24.026109+0000","last_fullsized":"2024-05-08T06:37:24.026109+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:33:43.783546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.037705+0000","last_change":"2024-05-08T06:37:23.033770+0000","last_active":"2024-05-08T06:37:23.037705+0000","last_peered":"2024-05-08T06:37:23.037705+0000","last_clean":"2024-05-08T06:37:23.037705+0000","last_became_active":"2024-05-08T06:37:23.033509+0000","last_became_peered":"2024-05-08T06:37:23.033509+0000","last_unstale":"2024-05-08T06:37:23.037705+0000","last_undegraded":"2024-05-08T06:37:23.037705+0000","last_fullsized":"2024-05-08T06:37:23.037705+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:44.020088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025824+0000","last_change":"2024-05-08T06:37:22.006162+0000","last_active":"2024-05-08T06:37:24.025824+0000","last_peered":"2024-05-08T06:37:24.025824+0000","last_clean":"2024-05-08T06:37:24.025824+0000","last_became_active":"2024-05-08T06:37:22.005887+0000","last_became_peered":"2024-05-08T06:37:22.005887+0000","last_unstale":"2024-05-08T06:37:24.025824+0000","last_undegraded":"2024-05-08T06:37:24.025824+0000","last_fullsized":"2024-05-08T06:37:24.025824+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:11:58.465794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.033714+0000","last_change":"2024-05-08T06:37:23.032276+0000","last_active":"2024-05-08T06:37:23.033714+0000","last_peered":"2024-05-08T06:37:23.033714+0000","last_clean":"2024-05-08T06:37:23.033714+0000","last_became_active":"2024-05-08T06:37:23.031981+0000","last_became_peered":"2024-05-08T06:37:23.031981+0000","last_unstale":"2024-05-08T06:37:23.033714+0000","last_undegraded":"2024-05-08T06:37:23.033714+0000","last_fullsized":"2024-05-08T06:37:23.033714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:56.252656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.024287+0000","last_change":"2024-05-08T06:37:22.010783+0000","last_active":"2024-05-08T06:37:24.024287+0000","last_peered":"2024-05-08T06:37:24.024287+0000","last_clean":"2024-05-08T06:37:24.024287+0000","last_became_active":"2024-05-08T06:37:22.010472+0000","last_became_peered":"2024-05-08T06:37:22.010472+0000","last_unstale":"2024-05-08T06:37:24.024287+0000","last_undegraded":"2024-05-08T06:37:24.024287+0000","last_fullsized":"2024-05-08T06:37:24.024287+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:52:39.204187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.029442+0000","last_change":"2024-05-08T06:37:23.028223+0000","last_active":"2024-05-08T06:37:23.029442+0000","last_peered":"2024-05-08T06:37:23.029442+0000","last_clean":"2024-05-08T06:37:23.029442+0000","last_became_active":"2024-05-08T06:37:23.027943+0000","last_became_peered":"2024-05-08T06:37:23.027943+0000","last_unstale":"2024-05-08T06:37:23.029442+0000","last_undegraded":"2024-05-08T06:37:23.029442+0000","last_fullsized":"2024-05-08T06:37:23.029442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:55.804695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025027+0000","last_change":"2024-05-08T06:37:22.009040+0000","last_active":"2024-05-08T06:37:24.025027+0000","last_peered":"2024-05-08T06:37:24.025027+0000","last_clean":"2024-05-08T06:37:24.025027+0000","last_became_active":"2024-05-08T06:37:22.008790+0000","last_became_peered":"2024-05-08T06:37:22.008790+0000","last_unstale":"2024-05-08T06:37:24.025027+0000","last_undegraded":"2024-05-08T06:37:24.025027+0000","last_fullsized":"2024-05-08T06:37:24.025027+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:58.710469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.029510+0000","last_change":"2024-05-08T06:37:23.027631+0000","last_active":"2024-05-08T06:37:23.029510+0000","last_peered":"2024-05-08T06:37:23.029510+0000","last_clean":"2024-05-08T06:37:23.029510+0000","last_became_active":"2024-05-08T06:37:23.027360+0000","last_became_peered":"2024-05-08T06:37:23.027360+0000","last_unstale":"2024-05-08T06:37:23.029510+0000","last_undegraded":"2024-05-08T06:37:23.029510+0000","last_fullsized":"2024-05-08T06:37:23.029510+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:45:49.327398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025344+0000","last_change":"2024-05-08T06:37:22.010798+0000","last_active":"2024-05-08T06:37:24.025344+0000","last_peered":"2024-05-08T06:37:24.025344+0000","last_clean":"2024-05-08T06:37:24.025344+0000","last_became_active":"2024-05-08T06:37:22.010534+0000","last_became_peered":"2024-05-08T06:37:22.010534+0000","last_unstale":"2024-05-08T06:37:24.025344+0000","last_undegraded":"2024-05-08T06:37:24.025344+0000","last_fullsized":"2024-05-08T06:37:24.025344+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:38:52.412302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.030980+0000","last_change":"2024-05-08T06:37:23.029182+0000","last_active":"2024-05-08T06:37:23.030980+0000","last_peered":"2024-05-08T06:37:23.030980+0000","last_clean":"2024-05-08T06:37:23.030980+0000","last_became_active":"2024-05-08T06:37:23.028885+0000","last_became_peered":"2024-05-08T06:37:23.028885+0000","last_unstale":"2024-05-08T06:37:23.030980+0000","last_undegraded":"2024-05-08T06:37:23.030980+0000","last_fullsized":"2024-05-08T06:37:23.030980+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:35.676798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.021640+0000","last_change":"2024-05-08T06:37:22.011615+0000","last_active":"2024-05-08T06:37:23.021640+0000","last_peered":"2024-05-08T06:37:23.021640+0000","last_clean":"2024-05-08T06:37:23.021640+0000","last_became_active":"2024-05-08T06:37:22.011285+0000","last_became_peered":"2024-05-08T06:37:22.011285+0000","last_unstale":"2024-05-08T06:37:23.021640+0000","last_undegraded":"2024-05-08T06:37:23.021640+0000","last_fullsized":"2024-05-08T06:37:23.021640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:16.525080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.033676+0000","last_change":"2024-05-08T06:37:23.031517+0000","last_active":"2024-05-08T06:37:23.033676+0000","last_peered":"2024-05-08T06:37:23.033676+0000","last_clean":"2024-05-08T06:37:23.033676+0000","last_became_active":"2024-05-08T06:37:23.031254+0000","last_became_peered":"2024-05-08T06:37:23.031254+0000","last_unstale":"2024-05-08T06:37:23.033676+0000","last_undegraded":"2024-05-08T06:37:23.033676+0000","last_fullsized":"2024-05-08T06:37:23.033676+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:04:31.106795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.035204+0000","last_change":"2024-05-08T06:37:22.011974+0000","last_active":"2024-05-08T06:37:24.035204+0000","last_peered":"2024-05-08T06:37:24.035204+0000","last_clean":"2024-05-08T06:37:24.035204+0000","last_became_active":"2024-05-08T06:37:22.011659+0000","last_became_peered":"2024-05-08T06:37:22.011659+0000","last_unstale":"2024-05-08T06:37:24.035204+0000","last_undegraded":"2024-05-08T06:37:24.035204+0000","last_fullsized":"2024-05-08T06:37:24.035204+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:29:02.885057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.039505+0000","last_change":"2024-05-08T06:37:23.034352+0000","last_active":"2024-05-08T06:37:23.039505+0000","last_peered":"2024-05-08T06:37:23.039505+0000","last_clean":"2024-05-08T06:37:23.039505+0000","last_became_active":"2024-05-08T06:37:23.033946+0000","last_became_peered":"2024-05-08T06:37:23.033946+0000","last_unstale":"2024-05-08T06:37:23.039505+0000","last_undegraded":"2024-05-08T06:37:23.039505+0000","last_fullsized":"2024-05-08T06:37:23.039505+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:16.749074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"31'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.034313+0000","last_change":"2024-05-08T06:37:22.010633+0000","last_active":"2024-05-08T06:37:24.034313+0000","last_peered":"2024-05-08T06:37:24.034313+0000","last_clean":"2024-05-08T06:37:24.034313+0000","last_became_active":"2024-05-08T06:37:22.010391+0000","last_became_peered":"2024-05-08T06:37:22.010391+0000","last_unstale":"2024-05-08T06:37:24.034313+0000","last_undegraded":"2024-05-08T06:37:24.034313+0000","last_fullsized":"2024-05-08T06:37:24.034313+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:13.378762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.024150+0000","last_change":"2024-05-08T06:37:23.018540+0000","last_active":"2024-05-08T06:37:24.024150+0000","last_peered":"2024-05-08T06:37:24.024150+0000","last_clean":"2024-05-08T06:37:24.024150+0000","last_became_active":"2024-05-08T06:37:23.018269+0000","last_became_peered":"2024-05-08T06:37:23.018269+0000","last_unstale":"2024-05-08T06:37:24.024150+0000","last_undegraded":"2024-05-08T06:37:24.024150+0000","last_fullsized":"2024-05-08T06:37:24.024150+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:49.479743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.035139+0000","last_change":"2024-05-08T06:37:22.013019+0000","last_active":"2024-05-08T06:37:24.035139+0000","last_peered":"2024-05-08T06:37:24.035139+0000","last_clean":"2024-05-08T06:37:24.035139+0000","last_became_active":"2024-05-08T06:37:22.012781+0000","last_became_peered":"2024-05-08T06:37:22.012781+0000","last_unstale":"2024-05-08T06:37:24.035139+0000","last_undegraded":"2024-05-08T06:37:24.035139+0000","last_fullsized":"2024-05-08T06:37:24.035139+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:36:35.031102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025080+0000","last_change":"2024-05-08T06:37:23.024421+0000","last_active":"2024-05-08T06:37:24.025080+0000","last_peered":"2024-05-08T06:37:24.025080+0000","last_clean":"2024-05-08T06:37:24.025080+0000","last_became_active":"2024-05-08T06:37:23.024131+0000","last_became_peered":"2024-05-08T06:37:23.024131+0000","last_unstale":"2024-05-08T06:37:24.025080+0000","last_undegraded":"2024-05-08T06:37:24.025080+0000","last_fullsized":"2024-05-08T06:37:24.025080+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:43:47.264013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.024382+0000","last_change":"2024-05-08T06:37:22.006496+0000","last_active":"2024-05-08T06:37:24.024382+0000","last_peered":"2024-05-08T06:37:24.024382+0000","last_clean":"2024-05-08T06:37:24.024382+0000","last_became_active":"2024-05-08T06:37:22.006216+0000","last_became_peered":"2024-05-08T06:37:22.006216+0000","last_unstale":"2024-05-08T06:37:24.024382+0000","last_undegraded":"2024-05-08T06:37:24.024382+0000","last_fullsized":"2024-05-08T06:37:24.024382+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:16.525080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.024297+0000","last_change":"2024-05-08T06:37:23.019090+0000","last_active":"2024-05-08T06:37:24.024297+0000","last_peered":"2024-05-08T06:37:24.024297+0000","last_clean":"2024-05-08T06:37:24.024297+0000","last_became_active":"2024-05-08T06:37:23.018812+0000","last_became_peered":"2024-05-08T06:37:23.018812+0000","last_unstale":"2024-05-08T06:37:24.024297+0000","last_undegraded":"2024-05-08T06:37:24.024297+0000","last_fullsized":"2024-05-08T06:37:24.024297+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:01.330678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.034805+0000","last_change":"2024-05-08T06:37:22.012365+0000","last_active":"2024-05-08T06:37:24.034805+0000","last_peered":"2024-05-08T06:37:24.034805+0000","last_clean":"2024-05-08T06:37:24.034805+0000","last_became_active":"2024-05-08T06:37:22.012059+0000","last_became_peered":"2024-05-08T06:37:22.012059+0000","last_unstale":"2024-05-08T06:37:24.034805+0000","last_undegraded":"2024-05-08T06:37:24.034805+0000","last_fullsized":"2024-05-08T06:37:24.034805+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:16.525080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.030906+0000","last_change":"2024-05-08T06:37:23.022907+0000","last_active":"2024-05-08T06:37:23.030906+0000","last_peered":"2024-05-08T06:37:23.030906+0000","last_clean":"2024-05-08T06:37:23.030906+0000","last_became_active":"2024-05-08T06:37:23.022652+0000","last_became_peered":"2024-05-08T06:37:23.022652+0000","last_unstale":"2024-05-08T06:37:23.030906+0000","last_undegraded":"2024-05-08T06:37:23.030906+0000","last_fullsized":"2024-05-08T06:37:23.030906+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:49.479743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025367+0000","last_change":"2024-05-08T06:37:22.005487+0000","last_active":"2024-05-08T06:37:24.025367+0000","last_peered":"2024-05-08T06:37:24.025367+0000","last_clean":"2024-05-08T06:37:24.025367+0000","last_became_active":"2024-05-08T06:37:22.005219+0000","last_became_peered":"2024-05-08T06:37:22.005219+0000","last_unstale":"2024-05-08T06:37:24.025367+0000","last_undegraded":"2024-05-08T06:37:24.025367+0000","last_fullsized":"2024-05-08T06:37:24.025367+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:49.134193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.024732+0000","last_change":"2024-05-08T06:37:23.028508+0000","last_active":"2024-05-08T06:37:24.024732+0000","last_peered":"2024-05-08T06:37:24.024732+0000","last_clean":"2024-05-08T06:37:24.024732+0000","last_became_active":"2024-05-08T06:37:23.027783+0000","last_became_peered":"2024-05-08T06:37:23.027783+0000","last_unstale":"2024-05-08T06:37:24.024732+0000","last_undegraded":"2024-05-08T06:37:24.024732+0000","last_fullsized":"2024-05-08T06:37:24.024732+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:59.042090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.016053+0000","last_change":"2024-05-08T06:37:22.012473+0000","last_active":"2024-05-08T06:37:23.016053+0000","last_peered":"2024-05-08T06:37:23.016053+0000","last_clean":"2024-05-08T06:37:23.016053+0000","last_became_active":"2024-05-08T06:37:22.012219+0000","last_became_peered":"2024-05-08T06:37:22.012219+0000","last_unstale":"2024-05-08T06:37:23.016053+0000","last_undegraded":"2024-05-08T06:37:23.016053+0000","last_fullsized":"2024-05-08T06:37:23.016053+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:58.035899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.038578+0000","last_change":"2024-05-08T06:37:23.032983+0000","last_active":"2024-05-08T06:37:23.038578+0000","last_peered":"2024-05-08T06:37:23.038578+0000","last_clean":"2024-05-08T06:37:23.038578+0000","last_became_active":"2024-05-08T06:37:23.032574+0000","last_became_peered":"2024-05-08T06:37:23.032574+0000","last_unstale":"2024-05-08T06:37:23.038578+0000","last_undegraded":"2024-05-08T06:37:23.038578+0000","last_fullsized":"2024-05-08T06:37:23.038578+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:21.989537+0000","last_clean_scrub_stamp":"2024-05-08T06:37:21.989537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:01.330678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-08T06:37:24.025046+0000","last_change":"2024-05-08T06:37:22.006187+0000","last_active":"2024-05-08T06:37:24.025046+0000","last_peered":"2024-05-08T06:37:24.025046+0000","last_clean":"2024-05-08T06:37:24.025046+0000","last_became_active":"2024-05-08T06:37:22.005919+0000","last_became_peered":"2024-05-08T06:37:22.005919+0000","last_unstale":"2024-05-08T06:37:24.025046+0000","last_undegraded":"2024-05-08T06:37:24.025046+0000","last_fullsized":"2024-05-08T06:37:24.025046+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:37:20.983346+0000","last_clean_scrub_stamp":"2024-05-08T06:37:20.983346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:15.248123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":128,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-08T06:37:23.024978+0000","last_change":"2024-05-08T06:36:31.710287+0000","last_active":"2024-05-08T06:37:23.024978+0000","last_peered":"2024-05-08T06:37:23.024978+0000","last_clean":"2024-05-08T06:37:23.024978+0000","last_became_active":"2024-05-08T06:36:31.709270+0000","last_became_peered":"2024-05-08T06:36:31.709270+0000","last_unstale":"2024-05-08T06:37:23.024978+0000","last_undegraded":"2024-05-08T06:37:23.024978+0000","last_fullsized":"2024-05-08T06:37:23.024978+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_clean_scrub_stamp":"2024-05-08T06:36:30.122918+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:42.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":933888,"data_stored":718284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":356352,"data_stored":134742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":344064,"data_stored":128498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":344064,"data_stored":127930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":129460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":339968,"data_stored":127896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":129460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":718194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:37:26.370 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T06:37:26.370 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T06:37:26.514 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.515+0000 7f9cd33e4640 1 Processor -- start 2024-05-08T06:37:26.515 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.515+0000 7f9cd33e4640 1 -- start start 2024-05-08T06:37:26.518 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.516+0000 7f9cd33e4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9ccc148d90 0x7f9ccc0d2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.518 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.516+0000 7f9cd33e4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9ccc0b6ac0 0x7f9ccc06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.518 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.516+0000 7f9cd33e4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ccc162fd0 con 0x7f9ccc0d34e0 2024-05-08T06:37:26.518 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.516+0000 7f9cd33e4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f9ccc0d3940 con 0x7f9ccc0b6ac0 2024-05-08T06:37:26.518 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.516+0000 7f9cd33e4640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9ccc0b7070 con 0x7f9ccc148d90 2024-05-08T06:37:26.518 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.516+0000 7f9cd1981640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9ccc0b6ac0 0x7f9ccc06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.518 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.516+0000 7f9cd097f640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f9ccc0d34e0 0x7f9ccc0b63b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33840/0 (socket says 172.21.15.163:33840) 2024-05-08T06:37:26.518 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.516+0000 7f9cd1981640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9ccc0b6ac0 0x7f9ccc06d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53594/0 (socket says 172.21.15.163:53594) 2024-05-08T06:37:26.518 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.516+0000 7f9cd097f640 1 -- 172.21.15.163:0/13394820 learned_addr learned my addr 172.21.15.163:0/13394820 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:26.518 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.517+0000 7f9cd1180640 1 --2- 172.21.15.163:0/13394820 >> v2:172.21.15.192:3300/0 conn(0x7f9ccc148d90 0x7f9ccc0d2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.518 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.517+0000 7f9cd1981640 1 -- 172.21.15.163:0/13394820 >> v2:172.21.15.192:3300/0 conn(0x7f9ccc148d90 msgr2=0x7f9ccc0d2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.517+0000 7f9cd1981640 1 --2- 172.21.15.163:0/13394820 >> v2:172.21.15.192:3300/0 conn(0x7f9ccc148d90 0x7f9ccc0d2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.517+0000 7f9cd1981640 1 -- 172.21.15.163:0/13394820 >> v1:172.21.15.163:6789/0 conn(0x7f9ccc0d34e0 legacy=0x7f9ccc0b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.517+0000 7f9ccbfff640 1 -- 172.21.15.163:0/13394820 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 665193952 0 0) 0x7f9ccc162fd0 con 0x7f9ccc0d34e0 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.517+0000 7f9cd1981640 1 -- 172.21.15.163:0/13394820 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc1632b0 con 0x7f9ccc0b6ac0 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.517+0000 7f9cd1981640 1 --2- 172.21.15.163:0/13394820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ccc0b6ac0 0x7f9ccc06d900 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc06a0a0 tx=0x7f9cc402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f30010994983d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.517+0000 7f9ccbfff640 1 -- 172.21.15.163:0/13394820 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4007740 con 0x7f9ccc0b6ac0 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.518+0000 7f9ccbfff640 1 -- 172.21.15.163:0/13394820 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc40078e0 con 0x7f9ccc0b6ac0 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.518+0000 7f9ccbfff640 1 -- 172.21.15.163:0/13394820 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4007cb0 con 0x7f9ccc0b6ac0 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.518+0000 7f9cd33e4640 1 -- 172.21.15.163:0/13394820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ccc0b6ac0 msgr2=0x7f9ccc06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.518+0000 7f9cd33e4640 1 --2- 172.21.15.163:0/13394820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ccc0b6ac0 0x7f9ccc06d900 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc06a0a0 tx=0x7f9cc402a5f0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.518+0000 7f9cd33e4640 1 -- 172.21.15.163:0/13394820 shutdown_connections 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.518+0000 7f9cd33e4640 1 --2- 172.21.15.163:0/13394820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ccc0b6ac0 0x7f9ccc06d900 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.518+0000 7f9cd33e4640 1 --2- 172.21.15.163:0/13394820 >> v2:172.21.15.192:3300/0 conn(0x7f9ccc148d90 0x7f9ccc0d2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.518+0000 7f9cd33e4640 1 -- 172.21.15.163:0/13394820 >> 172.21.15.163:0/13394820 conn(0x7f9ccc0b4da0 msgr2=0x7f9ccc0b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:26.519 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.518+0000 7f9cd33e4640 1 -- 172.21.15.163:0/13394820 shutdown_connections 2024-05-08T06:37:26.520 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.518+0000 7f9cd33e4640 1 -- 172.21.15.163:0/13394820 wait complete. 2024-05-08T06:37:26.522 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.519+0000 7f9cd33e4640 1 Processor -- start 2024-05-08T06:37:26.522 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.519+0000 7f9cd33e4640 1 -- start start 2024-05-08T06:37:26.522 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd33e4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ccc0b6ac0 0x7f9ccc15ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.522 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd33e4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ccc0d34e0 0x7f9ccc0d2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.522 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd33e4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ccc148d90 0x7f9ccc140e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.522 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd33e4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc163d90 con 0x7f9ccc148d90 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd33e4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc163870 con 0x7f9ccc0d34e0 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd33e4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9ccc06de40 con 0x7f9ccc0b6ac0 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd1981640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ccc148d90 0x7f9ccc140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd1981640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ccc148d90 0x7f9ccc140e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53604/0 (socket says 172.21.15.163:53604) 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd1981640 1 -- 172.21.15.163:0/3277573637 learned_addr learned my addr 172.21.15.163:0/3277573637 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd1180640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ccc0b6ac0 0x7f9ccc15ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd097f640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ccc0d34e0 0x7f9ccc0d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd1981640 1 -- 172.21.15.163:0/3277573637 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ccc0b6ac0 msgr2=0x7f9ccc15ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd1981640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ccc0b6ac0 0x7f9ccc15ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd1981640 1 -- 172.21.15.163:0/3277573637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ccc0d34e0 msgr2=0x7f9ccc0d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd1981640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ccc0d34e0 0x7f9ccc0d2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd1981640 1 -- 172.21.15.163:0/3277573637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc15ef50 con 0x7f9ccc148d90 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.520+0000 7f9cd1180640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ccc0b6ac0 0x7f9ccc15ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.521+0000 7f9cd097f640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ccc0d34e0 0x7f9ccc0d2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.521+0000 7f9cd1981640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ccc148d90 0x7f9ccc140e80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9cc402abe0 tx=0x7f9cc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:26.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.521+0000 7f9cc9ffb640 1 -- 172.21.15.163:0/3277573637 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4040070 con 0x7f9ccc148d90 2024-05-08T06:37:26.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.521+0000 7f9cc9ffb640 1 -- 172.21.15.163:0/3277573637 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc4031bc0 con 0x7f9ccc148d90 2024-05-08T06:37:26.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.521+0000 7f9cd33e4640 1 -- 172.21.15.163:0/3277573637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ccc142440 con 0x7f9ccc148d90 2024-05-08T06:37:26.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.521+0000 7f9cc9ffb640 1 -- 172.21.15.163:0/3277573637 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4045460 con 0x7f9ccc148d90 2024-05-08T06:37:26.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.521+0000 7f9cd33e4640 1 -- 172.21.15.163:0/3277573637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ccc13c760 con 0x7f9ccc148d90 2024-05-08T06:37:26.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.522+0000 7f9cd33e4640 1 -- 172.21.15.163:0/3277573637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ccc075dc0 con 0x7f9ccc148d90 2024-05-08T06:37:26.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.523+0000 7f9cc9ffb640 1 -- 172.21.15.163:0/3277573637 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9cc4039040 con 0x7f9ccc148d90 2024-05-08T06:37:26.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.523+0000 7f9cc9ffb640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9cb80762d0 0x7f9cb8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.523+0000 7f9cd1180640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9cb80762d0 0x7f9cb8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.523+0000 7f9cc9ffb640 1 -- 172.21.15.163:0/3277573637 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f9cc40b8e80 con 0x7f9ccc148d90 2024-05-08T06:37:26.524 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.524+0000 7f9cd1180640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9cb80762d0 0x7f9cb8078790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc006fd0 tx=0x7f9cbc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:26.527 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.528+0000 7f9cc9ffb640 1 -- 172.21.15.163:0/3277573637 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9cc4082870 con 0x7f9ccc148d90 2024-05-08T06:37:26.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.777+0000 7f9cd33e4640 1 -- 172.21.15.163:0/3277573637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9ccc06a550 con 0x7f9ccc148d90 2024-05-08T06:37:26.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.780+0000 7f9cc9ffb640 1 -- 172.21.15.163:0/3277573637 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f9cc4082170 con 0x7f9ccc148d90 2024-05-08T06:37:26.779 INFO:teuthology.orchestra.run.smithi163.stdout:[client.0] 2024-05-08T06:37:26.779 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQBAHTtmobegIxAApWbSIBzhwZQyiH9lGjNodA== 2024-05-08T06:37:26.779 INFO:teuthology.orchestra.run.smithi163.stdout: caps mds = "allow" 2024-05-08T06:37:26.779 INFO:teuthology.orchestra.run.smithi163.stdout: caps mon = "allow r" 2024-05-08T06:37:26.779 INFO:teuthology.orchestra.run.smithi163.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T06:37:26.779 INFO:teuthology.orchestra.run.smithi163.stderr:updated caps for client.0 2024-05-08T06:37:26.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.782+0000 7f9cd33e4640 1 -- 172.21.15.163:0/3277573637 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9cb80762d0 msgr2=0x7f9cb8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.782+0000 7f9cd33e4640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9cb80762d0 0x7f9cb8078790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc006fd0 tx=0x7f9cbc008040 comp rx=0 tx=0).stop 2024-05-08T06:37:26.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.782+0000 7f9cd33e4640 1 -- 172.21.15.163:0/3277573637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ccc148d90 msgr2=0x7f9ccc140e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.782+0000 7f9cd33e4640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ccc148d90 0x7f9ccc140e80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9cc402abe0 tx=0x7f9cc4004a40 comp rx=0 tx=0).stop 2024-05-08T06:37:26.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.782+0000 7f9cd33e4640 1 -- 172.21.15.163:0/3277573637 shutdown_connections 2024-05-08T06:37:26.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.782+0000 7f9cd33e4640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f9cb80762d0 0x7f9cb8078790 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.782+0000 7f9cd33e4640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ccc148d90 0x7f9ccc140e80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.782+0000 7f9cd33e4640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ccc0d34e0 0x7f9ccc0d2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.782+0000 7f9cd33e4640 1 --2- 172.21.15.163:0/3277573637 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ccc0b6ac0 0x7f9ccc15ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.782+0000 7f9cd33e4640 1 -- 172.21.15.163:0/3277573637 >> 172.21.15.163:0/3277573637 conn(0x7f9ccc0b4da0 msgr2=0x7f9ccc145990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:26.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.782+0000 7f9cd33e4640 1 -- 172.21.15.163:0/3277573637 shutdown_connections 2024-05-08T06:37:26.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.782+0000 7f9cd33e4640 1 -- 172.21.15.163:0/3277573637 wait complete. 2024-05-08T06:37:26.792 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:26.892 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.892+0000 7f0d24220640 1 Processor -- start 2024-05-08T06:37:26.892 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.893+0000 7f0d24220640 1 -- start start 2024-05-08T06:37:26.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.893+0000 7f0d24220640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d1c142810 0x7f0d1c142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.893+0000 7f0d24220640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0d1c1410b0 0x7f0d1c1414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.893+0000 7f0d24220640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d1c140450 con 0x7f0d1c140070 2024-05-08T06:37:26.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.893+0000 7f0d24220640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0d1c072e50 con 0x7f0d1c1410b0 2024-05-08T06:37:26.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.893+0000 7f0d24220640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1c072890 con 0x7f0d1c142810 2024-05-08T06:37:26.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.894+0000 7f0d227bd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0d1c1410b0 0x7f0d1c1414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.894+0000 7f0d217bb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d1c142810 0x7f0d1c142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.894+0000 7f0d227bd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0d1c1410b0 0x7f0d1c1414c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:55462/0 (socket says 172.21.15.163:55462) 2024-05-08T06:37:26.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.894+0000 7f0d21fbc640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f0d1c140070 0x7f0d1c071db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:34674/0 (socket says 172.21.15.163:34674) 2024-05-08T06:37:26.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.894+0000 7f0d227bd640 1 -- 172.21.15.163:0/809237678 learned_addr learned my addr 172.21.15.163:0/809237678 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:26.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.894+0000 7f0d227bd640 1 -- 172.21.15.163:0/809237678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d1c142810 msgr2=0x7f0d1c142ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:26.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.894+0000 7f0d227bd640 1 --2- 172.21.15.163:0/809237678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d1c142810 0x7f0d1c142ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.894+0000 7f0d227bd640 1 -- 172.21.15.163:0/809237678 >> v1:172.21.15.192:6789/0 conn(0x7f0d1c140070 legacy=0x7f0d1c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.894+0000 7f0d227bd640 1 -- 172.21.15.163:0/809237678 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c072b70 con 0x7f0d1c1410b0 2024-05-08T06:37:26.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.894+0000 7f0d217bb640 1 --2- 172.21.15.163:0/809237678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d1c142810 0x7f0d1c142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:26.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.895+0000 7f0d227bd640 1 --2- 172.21.15.163:0/809237678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d1c1410b0 0x7f0d1c1414c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009e50 tx=0x7f0d1402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1750ca43150569f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:26.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.895+0000 7f0d20fba640 1 -- 172.21.15.163:0/809237678 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d14036070 con 0x7f0d1c1410b0 2024-05-08T06:37:26.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.895+0000 7f0d20fba640 1 -- 172.21.15.163:0/809237678 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d14031600 con 0x7f0d1c1410b0 2024-05-08T06:37:26.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.895+0000 7f0d20fba640 1 -- 172.21.15.163:0/809237678 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d140319d0 con 0x7f0d1c1410b0 2024-05-08T06:37:26.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.895+0000 7f0d24220640 1 -- 172.21.15.163:0/809237678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d1c1410b0 msgr2=0x7f0d1c1414c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.895+0000 7f0d24220640 1 --2- 172.21.15.163:0/809237678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d1c1410b0 0x7f0d1c1414c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009e50 tx=0x7f0d1402a5f0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.896+0000 7f0d24220640 1 -- 172.21.15.163:0/809237678 shutdown_connections 2024-05-08T06:37:26.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.896+0000 7f0d24220640 1 --2- 172.21.15.163:0/809237678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d1c1410b0 0x7f0d1c1414c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.896+0000 7f0d24220640 1 --2- 172.21.15.163:0/809237678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d1c142810 0x7f0d1c142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.896+0000 7f0d24220640 1 -- 172.21.15.163:0/809237678 >> 172.21.15.163:0/809237678 conn(0x7f0d1c0b4da0 msgr2=0x7f0d1c0b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:26.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.896+0000 7f0d24220640 1 -- 172.21.15.163:0/809237678 shutdown_connections 2024-05-08T06:37:26.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.896+0000 7f0d24220640 1 -- 172.21.15.163:0/809237678 wait complete. 2024-05-08T06:37:26.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.897+0000 7f0d24220640 1 Processor -- start 2024-05-08T06:37:26.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.897+0000 7f0d24220640 1 -- start start 2024-05-08T06:37:26.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d24220640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d1c140070 0x7f0d1c1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d24220640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d1c1410b0 0x7f0d1c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d24220640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d1c142810 0x7f0d1c15c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d24220640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c0724c0 con 0x7f0d1c140070 2024-05-08T06:37:26.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d24220640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c1435b0 con 0x7f0d1c142810 2024-05-08T06:37:26.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d24220640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1c072640 con 0x7f0d1c1410b0 2024-05-08T06:37:26.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d21fbc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d1c140070 0x7f0d1c1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d21fbc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d1c140070 0x7f0d1c1563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53614/0 (socket says 172.21.15.163:53614) 2024-05-08T06:37:26.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d21fbc640 1 -- 172.21.15.163:0/3569000210 learned_addr learned my addr 172.21.15.163:0/3569000210 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:26.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d227bd640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d1c142810 0x7f0d1c15c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d21fbc640 1 -- 172.21.15.163:0/3569000210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d1c1410b0 msgr2=0x7f0d1c15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d217bb640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d1c1410b0 0x7f0d1c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d21fbc640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d1c1410b0 0x7f0d1c15c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d21fbc640 1 -- 172.21.15.163:0/3569000210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d1c142810 msgr2=0x7f0d1c15c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d21fbc640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d1c142810 0x7f0d1c15c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d21fbc640 1 -- 172.21.15.163:0/3569000210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c06e390 con 0x7f0d1c140070 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d227bd640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d1c142810 0x7f0d1c15c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d21fbc640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d1c140070 0x7f0d1c1563d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800c500 tx=0x7f0d1800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d127fc640 1 -- 172.21.15.163:0/3569000210 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18018070 con 0x7f0d1c140070 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d127fc640 1 -- 172.21.15.163:0/3569000210 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d18007500 con 0x7f0d1c140070 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d24220640 1 -- 172.21.15.163:0/3569000210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1c06e670 con 0x7f0d1c140070 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d127fc640 1 -- 172.21.15.163:0/3569000210 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d180049e0 con 0x7f0d1c140070 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.898+0000 7f0d24220640 1 -- 172.21.15.163:0/3569000210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1c06ec30 con 0x7f0d1c140070 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.899+0000 7f0d24220640 1 -- 172.21.15.163:0/3569000210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1c06a550 con 0x7f0d1c140070 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.900+0000 7f0d127fc640 1 -- 172.21.15.163:0/3569000210 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0d1801b070 con 0x7f0d1c140070 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.900+0000 7f0d127fc640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0d000762d0 0x7f0d00078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.901+0000 7f0d127fc640 1 -- 172.21.15.163:0/3569000210 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f0d180969a0 con 0x7f0d1c140070 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.901+0000 7f0d217bb640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0d000762d0 0x7f0d00078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:26.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.901+0000 7f0d217bb640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0d000762d0 0x7f0d00078790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009950 tx=0x7f0d0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:26.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:26.905+0000 7f0d127fc640 1 -- 172.21.15.163:0/3569000210 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0d18060390 con 0x7f0d1c140070 2024-05-08T06:37:27.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.126+0000 7f0d24220640 1 -- 172.21.15.163:0/3569000210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d1c0b5a40 con 0x7f0d1c140070 2024-05-08T06:37:27.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.126+0000 7f0d127fc640 1 -- 172.21.15.163:0/3569000210 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5190 (secure 0 0 0) 0x7f0d18068380 con 0x7f0d1c140070 2024-05-08T06:37:27.126 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:27.126 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1773142301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1773142301},{"type":"v1","addr":"172.21.15.163:6835","nonce":1773142301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2952930735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2952930735},{"type":"v1","addr":"172.21.15.192:6834","nonce":2952930735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1297249355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1297249355},{"type":"v1","addr":"172.21.15.192:6835","nonce":1297249355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:37:22.365453+0000","modified":"2024-05-08T06:37:24.368498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4732":{"gid":4732,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/880835577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":880835577},{"type":"v1","addr":"172.21.15.163:6837","nonce":880835577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4732,"qdb_cluster":[4732]},"id":2}]} 2024-05-08T06:37:27.126 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 14 2024-05-08T06:37:27.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.129+0000 7f0d24220640 1 -- 172.21.15.163:0/3569000210 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0d000762d0 msgr2=0x7f0d00078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.129+0000 7f0d24220640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0d000762d0 0x7f0d00078790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009950 tx=0x7f0d0c008040 comp rx=0 tx=0).stop 2024-05-08T06:37:27.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.129+0000 7f0d24220640 1 -- 172.21.15.163:0/3569000210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d1c140070 msgr2=0x7f0d1c1563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.129+0000 7f0d24220640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d1c140070 0x7f0d1c1563d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800c500 tx=0x7f0d1800c9c0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.129+0000 7f0d24220640 1 -- 172.21.15.163:0/3569000210 shutdown_connections 2024-05-08T06:37:27.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.129+0000 7f0d24220640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0d000762d0 0x7f0d00078790 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.129+0000 7f0d24220640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d1c142810 0x7f0d1c15c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.129+0000 7f0d24220640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d1c1410b0 0x7f0d1c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.129+0000 7f0d24220640 1 --2- 172.21.15.163:0/3569000210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d1c140070 0x7f0d1c1563d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.129+0000 7f0d24220640 1 -- 172.21.15.163:0/3569000210 >> 172.21.15.163:0/3569000210 conn(0x7f0d1c0b4da0 msgr2=0x7f0d1c156a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:27.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.129+0000 7f0d24220640 1 -- 172.21.15.163:0/3569000210 shutdown_connections 2024-05-08T06:37:27.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.129+0000 7f0d24220640 1 -- 172.21.15.163:0/3569000210 wait complete. 2024-05-08T06:37:27.541 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T06:37:22.365453+0000', 'modified': '2024-05-08T06:37:24.368498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4732}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4732': {'gid': 4732, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6837/880835577', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 880835577}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 880835577}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4732, 'qdb_cluster': [4732]} 2024-05-08T06:37:27.541 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T06:37:27.542 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T06:37:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.647+0000 7ffbb93c1640 1 Processor -- start 2024-05-08T06:37:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.647+0000 7ffbb93c1640 1 -- start start 2024-05-08T06:37:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb93c1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbb40b8240 0x7ffbb40d3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb93c1640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ffbb40d2250 0x7ffbb40d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb93c1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb40b8a60 con 0x7ffbb415f350 2024-05-08T06:37:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb93c1640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb40b9670 con 0x7ffbb40d2250 2024-05-08T06:37:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb93c1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb40b8d40 con 0x7ffbb40b8240 2024-05-08T06:37:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb2ffd640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ffbb415f350 0x7ffbb40bace0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33852/0 (socket says 172.21.15.163:33852) 2024-05-08T06:37:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb2ffd640 1 -- 172.21.15.163:0/1717205244 learned_addr learned my addr 172.21.15.163:0/1717205244 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/1717205244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbb40b8240 0x7ffbb40d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb37fe640 1 --2- 172.21.15.163:0/1717205244 >> v2:172.21.15.192:3300/0 conn(0x7ffbb40d2250 0x7ffbb40d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb27fc640 1 -- 172.21.15.163:0/1717205244 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2640991675 0 0) 0x7ffbb40b8a60 con 0x7ffbb415f350 2024-05-08T06:37:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb27fc640 1 -- 172.21.15.163:0/1717205244 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffba0003520 con 0x7ffbb415f350 2024-05-08T06:37:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb27fc640 1 -- 172.21.15.163:0/1717205244 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2957193874 0 0) 0x7ffba0003520 con 0x7ffbb415f350 2024-05-08T06:37:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb27fc640 1 -- 172.21.15.163:0/1717205244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbb40b8240 msgr2=0x7ffbb40d3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb27fc640 1 --2- 172.21.15.163:0/1717205244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbb40b8240 0x7ffbb40d3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb27fc640 1 -- 172.21.15.163:0/1717205244 >> v2:172.21.15.192:3300/0 conn(0x7ffbb40d2250 msgr2=0x7ffbb40d26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb27fc640 1 --2- 172.21.15.163:0/1717205244 >> v2:172.21.15.192:3300/0 conn(0x7ffbb40d2250 0x7ffbb40d26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.648+0000 7ffbb27fc640 1 -- 172.21.15.163:0/1717205244 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb40b9020 con 0x7ffbb415f350 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb27fc640 1 -- 172.21.15.163:0/1717205244 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7ffba40016f0 con 0x7ffbb415f350 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb27fc640 1 -- 172.21.15.163:0/1717205244 >> v1:172.21.15.163:6789/0 conn(0x7ffbb415f350 legacy=0x7ffbb40bace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb27fc640 1 --2- 172.21.15.163:0/1717205244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffba0003a00 0x7ffba0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb27fc640 1 --2- 172.21.15.163:0/1717205244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffba0004800 0x7ffba0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb27fc640 1 --2- 172.21.15.163:0/1717205244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbb40b8240 0x7ffba00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb27fc640 1 -- 172.21.15.163:0/1717205244 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40b9670 con 0x7ffba0003a00 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb27fc640 1 -- 172.21.15.163:0/1717205244 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40b8d40 con 0x7ffba0004800 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb27fc640 1 -- 172.21.15.163:0/1717205244 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffba000ba40 con 0x7ffbb40b8240 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/1717205244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffba0003a00 0x7ffba0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/1717205244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffba0004800 0x7ffba0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb37fe640 1 --2- 172.21.15.163:0/1717205244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbb40b8240 0x7ffba00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb2ffd640 1 -- 172.21.15.163:0/1717205244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbb40b8240 msgr2=0x7ffba00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/1717205244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbb40b8240 0x7ffba00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb2ffd640 1 -- 172.21.15.163:0/1717205244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffba0004800 msgr2=0x7ffba0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:27.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/1717205244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffba0004800 0x7ffba0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbb2ffd640 1 -- 172.21.15.163:0/1717205244 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba0003520 con 0x7ffba0003a00 2024-05-08T06:37:27.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.649+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/1717205244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffba0004800 0x7ffba0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:27.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.650+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/1717205244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffba0003a00 0x7ffba0003e20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffba400c210 tx=0x7ffba0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c048d790df1fee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:27.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.650+0000 7ffbb27fc640 1 -- 172.21.15.163:0/1717205244 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4007d30 con 0x7ffba0003a00 2024-05-08T06:37:27.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.650+0000 7ffbb27fc640 1 -- 172.21.15.163:0/1717205244 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba402e070 con 0x7ffba0003a00 2024-05-08T06:37:27.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.650+0000 7ffbb27fc640 1 -- 172.21.15.163:0/1717205244 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4005d30 con 0x7ffba0003a00 2024-05-08T06:37:27.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.650+0000 7ffbb93c1640 1 -- 172.21.15.163:0/1717205244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffba0003a00 msgr2=0x7ffba0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.650+0000 7ffbb93c1640 1 --2- 172.21.15.163:0/1717205244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffba0003a00 0x7ffba0003e20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffba400c210 tx=0x7ffba0005830 comp rx=0 tx=0).stop 2024-05-08T06:37:27.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.650+0000 7ffbb37fe640 1 -- 172.21.15.163:0/1717205244 reap_dead start 2024-05-08T06:37:27.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.650+0000 7ffbb93c1640 1 -- 172.21.15.163:0/1717205244 shutdown_connections 2024-05-08T06:37:27.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.650+0000 7ffbb93c1640 1 -- 172.21.15.163:0/1717205244 >> 172.21.15.163:0/1717205244 conn(0x7ffbb4073e70 msgr2=0x7ffbb4074280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:27.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.650+0000 7ffbb93c1640 1 -- 172.21.15.163:0/1717205244 shutdown_connections 2024-05-08T06:37:27.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.650+0000 7ffbb93c1640 1 -- 172.21.15.163:0/1717205244 wait complete. 2024-05-08T06:37:27.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.651+0000 7ffbb93c1640 1 Processor -- start 2024-05-08T06:37:27.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.652+0000 7ffbb93c1640 1 -- start start 2024-05-08T06:37:27.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.652+0000 7ffbb93c1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffba0003a00 0x7ffbb41609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.652+0000 7ffbb93c1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbb4160f20 0x7ffbb40caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.652+0000 7ffbb93c1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbb40caf60 0x7ffbb40c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.652+0000 7ffbb93c1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40b86a0 con 0x7ffbb4160f20 2024-05-08T06:37:27.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.652+0000 7ffbb93c1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb415f730 con 0x7ffbb40caf60 2024-05-08T06:37:27.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.652+0000 7ffbb93c1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb40b8820 con 0x7ffba0003a00 2024-05-08T06:37:27.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.652+0000 7ffbaa7fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbb4160f20 0x7ffbb40caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbaa7fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbb4160f20 0x7ffbb40caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53636/0 (socket says 172.21.15.163:53636) 2024-05-08T06:37:27.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbaa7fc640 1 -- 172.21.15.163:0/191439883 learned_addr learned my addr 172.21.15.163:0/191439883 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:27.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffba0003a00 0x7ffbb41609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbb37fe640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbb40caf60 0x7ffbb40c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbaa7fc640 1 -- 172.21.15.163:0/191439883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffba0003a00 msgr2=0x7ffbb41609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffba0003a00 0x7ffbb41609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbaa7fc640 1 -- 172.21.15.163:0/191439883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbb40caf60 msgr2=0x7ffbb40c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbb40caf60 0x7ffbb40c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbaa7fc640 1 -- 172.21.15.163:0/191439883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb41f58f0 con 0x7ffbb4160f20 2024-05-08T06:37:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffba0003a00 0x7ffbb41609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbb4160f20 0x7ffbb40caa20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c011570 tx=0x7ffb9c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbabfff640 1 -- 172.21.15.163:0/191439883 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c015870 con 0x7ffbb4160f20 2024-05-08T06:37:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbabfff640 1 -- 172.21.15.163:0/191439883 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c015a10 con 0x7ffbb4160f20 2024-05-08T06:37:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbb93c1640 1 -- 172.21.15.163:0/191439883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb41f5bd0 con 0x7ffbb4160f20 2024-05-08T06:37:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbabfff640 1 -- 172.21.15.163:0/191439883 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c015dc0 con 0x7ffbb4160f20 2024-05-08T06:37:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.653+0000 7ffbb93c1640 1 -- 172.21.15.163:0/191439883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb41f55a0 con 0x7ffbb4160f20 2024-05-08T06:37:27.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.655+0000 7ffbabfff640 1 -- 172.21.15.163:0/191439883 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffb9c01d400 con 0x7ffbb4160f20 2024-05-08T06:37:27.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.656+0000 7ffbabfff640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffb94076300 0x7ffb940787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.656+0000 7ffbabfff640 1 -- 172.21.15.163:0/191439883 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7ffb9c097d60 con 0x7ffbb4160f20 2024-05-08T06:37:27.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.656+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffb94076300 0x7ffb940787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.656+0000 7ffbbc1b4740 1 Processor -- start 2024-05-08T06:37:27.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.656+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffb94076300 0x7ffb940787c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffba4032000 tx=0x7ffba4007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:27.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.657+0000 7ffbbc1b4740 1 -- start start 2024-05-08T06:37:27.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.657+0000 7ffbbc1b4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b4700f50 0x5619b4703760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.657+0000 7ffbbc1b4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4701330 0x5619b4703ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.666 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.657+0000 7ffbbc1b4740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b46d3bb0 0x5619b46d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.657+0000 7ffbbc1b4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5619b463d8d0 con 0x5619b4701330 2024-05-08T06:37:27.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.657+0000 7ffbbc1b4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5619b45f6c10 con 0x5619b4700f50 2024-05-08T06:37:27.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.657+0000 7ffbbc1b4740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5619b4639890 con 0x5619b46d3bb0 2024-05-08T06:37:27.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.657+0000 7ffbaa7fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4701330 0x5619b4703ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.657+0000 7ffbaa7fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4701330 0x5619b4703ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53652/0 (socket says 172.21.15.163:53652) 2024-05-08T06:37:27.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.657+0000 7ffbaa7fc640 1 -- 172.21.15.163:0/778794534 learned_addr learned my addr 172.21.15.163:0/778794534 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:27.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.658+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/778794534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b4700f50 0x5619b4703760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.658+0000 7ffbb37fe640 1 --2- 172.21.15.163:0/778794534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b46d3bb0 0x5619b46d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.658+0000 7ffbaa7fc640 1 -- 172.21.15.163:0/778794534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b46d3bb0 msgr2=0x5619b46d73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:27.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.658+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/778794534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b46d3bb0 0x5619b46d73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.658+0000 7ffbaa7fc640 1 -- 172.21.15.163:0/778794534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b4700f50 msgr2=0x5619b4703760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:27.667 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.658+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/778794534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b4700f50 0x5619b4703760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.658+0000 7ffbaa7fc640 1 -- 172.21.15.163:0/778794534 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619b45fe5d0 con 0x5619b4701330 2024-05-08T06:37:27.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.658+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/778794534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4701330 0x5619b4703ca0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c096340 tx=0x7ffb9c060f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac9aa1e105935dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:27.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.658+0000 7ffba8ff9640 1 -- 172.21.15.163:0/778794534 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c07a020 con 0x5619b4701330 2024-05-08T06:37:27.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.658+0000 7ffba8ff9640 1 -- 172.21.15.163:0/778794534 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c065400 con 0x5619b4701330 2024-05-08T06:37:27.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.658+0000 7ffba8ff9640 1 -- 172.21.15.163:0/778794534 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c024d80 con 0x5619b4701330 2024-05-08T06:37:27.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.659+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/778794534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4701330 msgr2=0x5619b4703ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.659+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/778794534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4701330 0x5619b4703ca0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c096340 tx=0x7ffb9c060f50 comp rx=0 tx=0).stop 2024-05-08T06:37:27.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.659+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/778794534 shutdown_connections 2024-05-08T06:37:27.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.659+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/778794534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b46d3bb0 0x5619b46d73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.668 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.659+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/778794534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4701330 0x5619b4703ca0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.659+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/778794534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b4700f50 0x5619b4703760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.659+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/778794534 >> 172.21.15.163:0/778794534 conn(0x5619b46d5e20 msgr2=0x5619b471e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.659+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/778794534 shutdown_connections 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.659+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/778794534 wait complete. 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.659+0000 7ffbbc1b4740 1 -- start start 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.659+0000 7ffbbc1b4740 1 Processor -- start 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbbc1b4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4700f50 0x5619b4704ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbbc1b4740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b4701330 0x5619b47192b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbbc1b4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b46fdaf0 0x5619b47197f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbbc1b4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5619b46d8fd0 con 0x5619b4700f50 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbbc1b4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5619b45fa760 con 0x5619b46fdaf0 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbbc1b4740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5619b46d92b0 con 0x5619b4701330 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbbc1b4740 10 client.0 _ll_register_callbacks cb 0x5619b4633cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbb2ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4700f50 0x5619b4704ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.669 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbb2ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4700f50 0x5619b4704ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53668/0 (socket says 172.21.15.163:53668) 2024-05-08T06:37:27.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbb2ffd640 1 -- 172.21.15.163:0/2643010220 learned_addr learned my addr 172.21.15.163:0/2643010220 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:27.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/2643010220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b4701330 0x5619b47192b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbb37fe640 1 --2- 172.21.15.163:0/2643010220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b46fdaf0 0x5619b47197f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbb2ffd640 1 -- 172.21.15.163:0/2643010220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b4701330 msgr2=0x5619b47192b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:27.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/2643010220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b4701330 0x5619b47192b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbb2ffd640 1 -- 172.21.15.163:0/2643010220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b46fdaf0 msgr2=0x5619b47197f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:27.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/2643010220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b46fdaf0 0x5619b47197f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.660+0000 7ffbb2ffd640 1 -- 172.21.15.163:0/2643010220 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619b434f2d0 con 0x5619b4700f50 2024-05-08T06:37:27.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/2643010220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4700f50 0x5619b4704ce0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffba400c330 tx=0x7ffba402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9a2e8156608288 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:27.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffb91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:37:27.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffb91ffb640 1 -- 172.21.15.163:0/2643010220 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4052060 con 0x5619b4700f50 2024-05-08T06:37:27.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffb91ffb640 1 -- 172.21.15.163:0/2643010220 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7ffba404d480 con 0x5619b4700f50 2024-05-08T06:37:27.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/2643010220 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb9c065400 con 0x5619b4700f50 2024-05-08T06:37:27.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffb91ffb640 1 -- 172.21.15.163:0/2643010220 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7ffba4058050 con 0x5619b4700f50 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffbbc1b4740 10 client.4935 fetch_fsmap learned FSMap version 14 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/2643010220 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x5619b44196b0 con 0x5619b4700f50 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffb91ffb640 1 -- 172.21.15.163:0/2643010220 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c065400 con 0x5619b4700f50 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffb91ffb640 1 -- 172.21.15.163:0/2643010220 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7ffba405b020 con 0x5619b4700f50 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffbbc1b4740 10 client.4935 fetch_fsmap finished waiting for FSMap version 14 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffbbc1b4740 10 client.4935 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffbbc1b4740 20 client.4935 populate_metadata read hostname 'smithi163' 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/2643010220 >> [v2:172.21.15.163:6836/880835577,v1:172.21.15.163:6837/880835577] conn(0x5619b4720070 0x5619b473d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffbbc1b4740 4 client.4935 mds_command: new command op to 4732 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/2643010220 --> [v2:172.21.15.163:6836/880835577,v1:172.21.15.163:6837/880835577] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619b45c1760 con 0x5619b4720070 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.661+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/2643010220 >> [v2:172.21.15.163:6836/880835577,v1:172.21.15.163:6837/880835577] conn(0x5619b4720070 0x5619b473d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffb91ffb640 10 client.4935 ms_handle_connect on v2:172.21.15.163:6836/880835577 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/2643010220 >> [v2:172.21.15.163:6836/880835577,v1:172.21.15.163:6837/880835577] conn(0x5619b4720070 0x5619b473d560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1283b60440373f72 server_cookie=5d15be9d88d5181b in_seq=0 out_seq=0 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffb91ffb640 1 -- 172.21.15.163:0/2643010220 <== mds.0 v2:172.21.15.163:6836/880835577 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5619b45c1760 con 0x5619b4720070 2024-05-08T06:37:27.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffb91ffb640 10 client.4935 handle_command_reply: tid=0 2024-05-08T06:37:27.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffbbc1b4740 1 client.4935 shutdown 2024-05-08T06:37:27.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/2643010220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4700f50 msgr2=0x5619b4704ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/2643010220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4700f50 0x5619b4704ce0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffba400c330 tx=0x7ffba402c070 comp rx=0 tx=0).stop 2024-05-08T06:37:27.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/2643010220 shutdown_connections 2024-05-08T06:37:27.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/2643010220 >> [v2:172.21.15.163:6836/880835577,v1:172.21.15.163:6837/880835577] conn(0x5619b4720070 0x5619b473d560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/2643010220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b46fdaf0 0x5619b47197f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/2643010220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b4701330 0x5619b47192b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffb91ffb640 0 client.4935 ms_handle_reset on v2:172.21.15.163:6836/880835577 2024-05-08T06:37:27.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/2643010220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4700f50 0x5619b4704ce0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/2643010220 >> 172.21.15.163:0/2643010220 conn(0x5619b46d3bb0 msgr2=0x5619b46d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:27.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/2643010220 shutdown_connections 2024-05-08T06:37:27.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.662+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/2643010220 wait complete. 2024-05-08T06:37:27.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.663+0000 7ffbbc1b4740 20 client.4935 trim_cache size 0 max 16384 2024-05-08T06:37:27.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.681+0000 7ffbbc1b4740 1 Processor -- start 2024-05-08T06:37:27.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.681+0000 7ffbbc1b4740 1 -- start start 2024-05-08T06:37:27.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.681+0000 7ffbbc1b4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b46fdaf0 0x5619b4703ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.681+0000 7ffbbc1b4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4700f50 0x5619b4721960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.681+0000 7ffbbc1b4740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b4701330 0x5619b4719100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.681+0000 7ffbbc1b4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb42010f0 con 0x5619b4700f50 2024-05-08T06:37:27.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.681+0000 7ffbbc1b4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40cce80 con 0x5619b46fdaf0 2024-05-08T06:37:27.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.681+0000 7ffbbc1b4740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb40cd4d0 con 0x5619b4701330 2024-05-08T06:37:27.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.682+0000 7ffbaa7fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4700f50 0x5619b4721960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.682+0000 7ffbaa7fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4700f50 0x5619b4721960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53678/0 (socket says 172.21.15.163:53678) 2024-05-08T06:37:27.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.682+0000 7ffbaa7fc640 1 -- 172.21.15.163:0/72707858 learned_addr learned my addr 172.21.15.163:0/72707858 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:27.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.682+0000 7ffbaa7fc640 1 -- 172.21.15.163:0/72707858 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b4701330 msgr2=0x5619b4719100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:27.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.682+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/72707858 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b4701330 0x5619b4719100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.682+0000 7ffbaa7fc640 1 -- 172.21.15.163:0/72707858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b46fdaf0 msgr2=0x5619b4703ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:27.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.682+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/72707858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b46fdaf0 0x5619b4703ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.682+0000 7ffbaa7fc640 1 -- 172.21.15.163:0/72707858 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619b4721ea0 con 0x5619b4700f50 2024-05-08T06:37:27.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.682+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/72707858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4700f50 0x5619b4721960 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0962c0 tx=0x7ffb9c05d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1f65525beedda6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:27.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.682+0000 7ffb93fff640 1 -- 172.21.15.163:0/72707858 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c07a040 con 0x5619b4700f50 2024-05-08T06:37:27.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.682+0000 7ffb93fff640 1 -- 172.21.15.163:0/72707858 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c024b80 con 0x5619b4700f50 2024-05-08T06:37:27.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.682+0000 7ffb93fff640 1 -- 172.21.15.163:0/72707858 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0602b0 con 0x5619b4700f50 2024-05-08T06:37:27.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.682+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/72707858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4700f50 msgr2=0x5619b4721960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.683+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/72707858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4700f50 0x5619b4721960 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0962c0 tx=0x7ffb9c05d040 comp rx=0 tx=0).stop 2024-05-08T06:37:27.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.683+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/72707858 shutdown_connections 2024-05-08T06:37:27.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.683+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/72707858 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b4701330 0x5619b4719100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.683+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/72707858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4700f50 0x5619b4721960 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.683+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/72707858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b46fdaf0 0x5619b4703ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.683+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/72707858 >> 172.21.15.163:0/72707858 conn(0x5619b46d3bb0 msgr2=0x5619b471b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:27.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.683+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/72707858 shutdown_connections 2024-05-08T06:37:27.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.683+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/72707858 wait complete. 2024-05-08T06:37:27.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.683+0000 7ffbbc1b4740 1 -- start start 2024-05-08T06:37:27.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.683+0000 7ffbbc1b4740 1 Processor -- start 2024-05-08T06:37:27.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbbc1b4740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b46fdaf0 0x5619b47a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbbc1b4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b4700f50 0x5619b46fb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbbc1b4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4701330 0x5619b46fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbbc1b4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5619b4720fe0 con 0x5619b4701330 2024-05-08T06:37:27.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbbc1b4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5619b471b2a0 con 0x5619b4700f50 2024-05-08T06:37:27.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbbc1b4740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5619b471e150 con 0x5619b46fdaf0 2024-05-08T06:37:27.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbbc1b4740 10 client.0 _ll_register_callbacks cb 0x7ffba00036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:37:27.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbb37fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4701330 0x5619b46fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbb37fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4701330 0x5619b46fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53684/0 (socket says 172.21.15.163:53684) 2024-05-08T06:37:27.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbb37fe640 1 -- 172.21.15.163:0/1914967475 learned_addr learned my addr 172.21.15.163:0/1914967475 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:27.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/1914967475 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b46fdaf0 0x5619b47a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbaa7fc640 1 --2- 172.21.15.163:0/1914967475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b4700f50 0x5619b46fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbb37fe640 1 -- 172.21.15.163:0/1914967475 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b46fdaf0 msgr2=0x5619b47a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:27.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbb37fe640 1 --2- 172.21.15.163:0/1914967475 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b46fdaf0 0x5619b47a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbb37fe640 1 -- 172.21.15.163:0/1914967475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b4700f50 msgr2=0x5619b46fb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:27.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbb37fe640 1 --2- 172.21.15.163:0/1914967475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b4700f50 0x5619b46fb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.684+0000 7ffbb37fe640 1 -- 172.21.15.163:0/1914967475 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619b4717740 con 0x5619b4701330 2024-05-08T06:37:27.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffbb37fe640 1 --2- 172.21.15.163:0/1914967475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4701330 0x5619b46fbc40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40d21a0 tx=0x7ffbac016710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173d2bf7052e73e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:27.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffb91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:37:27.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffb91ffb640 1 -- 172.21.15.163:0/1914967475 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac01c070 con 0x5619b4701330 2024-05-08T06:37:27.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffb91ffb640 1 -- 172.21.15.163:0/1914967475 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7ffbac021460 con 0x5619b4701330 2024-05-08T06:37:27.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/1914967475 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb9c024b80 con 0x5619b4701330 2024-05-08T06:37:27.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffb91ffb640 1 -- 172.21.15.163:0/1914967475 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7ffbac029050 con 0x5619b4701330 2024-05-08T06:37:27.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffbbc1b4740 10 client.4941 fetch_fsmap learned FSMap version 14 2024-05-08T06:37:27.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/1914967475 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x5619b4719a60 con 0x5619b4701330 2024-05-08T06:37:27.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffb91ffb640 1 -- 172.21.15.163:0/1914967475 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c024b80 con 0x5619b4701330 2024-05-08T06:37:27.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffb91ffb640 1 -- 172.21.15.163:0/1914967475 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7ffbac02c020 con 0x5619b4701330 2024-05-08T06:37:27.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffbbc1b4740 10 client.4941 fetch_fsmap finished waiting for FSMap version 14 2024-05-08T06:37:27.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffbbc1b4740 10 client.4941 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T06:37:27.690 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffbbc1b4740 20 client.4941 populate_metadata read hostname 'smithi163' 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/1914967475 >> [v2:172.21.15.163:6836/880835577,v1:172.21.15.163:6837/880835577] conn(0x5619b47a54d0 0x5619b47a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffbbc1b4740 4 client.4941 mds_command: new command op to 4732 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/1914967475 --> [v2:172.21.15.163:6836/880835577,v1:172.21.15.163:6837/880835577] -- command(tid 0: {"prefix": "status"}) -- 0x5619b4722540 con 0x5619b47a54d0 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/1914967475 >> [v2:172.21.15.163:6836/880835577,v1:172.21.15.163:6837/880835577] conn(0x5619b47a54d0 0x5619b47a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffb91ffb640 10 client.4941 ms_handle_connect on v2:172.21.15.163:6836/880835577 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.685+0000 7ffbb2ffd640 1 --2- 172.21.15.163:0/1914967475 >> [v2:172.21.15.163:6836/880835577,v1:172.21.15.163:6837/880835577] conn(0x5619b47a54d0 0x5619b47a58b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b5d79fb3e73a1d server_cookie=dddc20ee467f0481 in_seq=0 out_seq=0 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffb91ffb640 1 -- 172.21.15.163:0/1914967475 <== mds.0 v2:172.21.15.163:6836/880835577 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5619b4722540 con 0x5619b47a54d0 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffb91ffb640 10 client.4941 handle_command_reply: tid=0 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbbc1b4740 1 client.4941 shutdown 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/1914967475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4701330 msgr2=0x5619b46fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/1914967475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4701330 0x5619b46fbc40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40d21a0 tx=0x7ffbac016710 comp rx=0 tx=0).stop 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/1914967475 shutdown_connections 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/1914967475 >> [v2:172.21.15.163:6836/880835577,v1:172.21.15.163:6837/880835577] conn(0x5619b47a54d0 0x5619b47a58b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/1914967475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5619b4701330 0x5619b46fbc40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.691 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/1914967475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619b4700f50 0x5619b46fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbbc1b4740 1 --2- 172.21.15.163:0/1914967475 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5619b46fdaf0 0x5619b47a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffb91ffb640 0 client.4941 ms_handle_reset on v2:172.21.15.163:6836/880835577 2024-05-08T06:37:27.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/1914967475 >> 172.21.15.163:0/1914967475 conn(0x5619b46d3bb0 msgr2=0x5619b4719100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:27.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/1914967475 shutdown_connections 2024-05-08T06:37:27.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbbc1b4740 1 -- 172.21.15.163:0/1914967475 wait complete. 2024-05-08T06:37:27.692 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbbc1b4740 20 client.4941 trim_cache size 0 max 16384 2024-05-08T06:37:27.692 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:37:27.692 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "e639506e-cf71-49c6-ae4d-5af4601c36e0", 2024-05-08T06:37:27.692 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2024-05-08T06:37:27.692 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 4732, 2024-05-08T06:37:27.692 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2024-05-08T06:37:27.692 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2024-05-08T06:37:27.693 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2024-05-08T06:37:27.693 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 4.6674003759999998, 2024-05-08T06:37:27.693 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 14, 2024-05-08T06:37:27.693 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 31, 2024-05-08T06:37:27.693 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 31, 2024-05-08T06:37:27.693 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 11.723590807000001 2024-05-08T06:37:27.693 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:37:27.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbb93c1640 1 -- 172.21.15.163:0/191439883 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffb94076300 msgr2=0x7ffb940787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbb93c1640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffb94076300 0x7ffb940787c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffba4032000 tx=0x7ffba4007000 comp rx=0 tx=0).stop 2024-05-08T06:37:27.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbb93c1640 1 -- 172.21.15.163:0/191439883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbb4160f20 msgr2=0x7ffbb40caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbb93c1640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbb4160f20 0x7ffbb40caa20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c011570 tx=0x7ffb9c011a30 comp rx=0 tx=0).stop 2024-05-08T06:37:27.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbb93c1640 1 -- 172.21.15.163:0/191439883 shutdown_connections 2024-05-08T06:37:27.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbb93c1640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ffb94076300 0x7ffb940787c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbb93c1640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbb40caf60 0x7ffbb40c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbb93c1640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbb4160f20 0x7ffbb40caa20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbb93c1640 1 --2- 172.21.15.163:0/191439883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffba0003a00 0x7ffbb41609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.686+0000 7ffbb93c1640 1 -- 172.21.15.163:0/191439883 >> 172.21.15.163:0/191439883 conn(0x7ffbb4073e70 msgr2=0x7ffbb40d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:27.694 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.687+0000 7ffbb93c1640 1 -- 172.21.15.163:0/191439883 shutdown_connections 2024-05-08T06:37:27.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.687+0000 7ffbb93c1640 1 -- 172.21.15.163:0/191439883 wait complete. 2024-05-08T06:37:27.697 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T06:37:27.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.799+0000 7f7ef6b2b640 1 Processor -- start 2024-05-08T06:37:27.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.799+0000 7f7ef6b2b640 1 -- start start 2024-05-08T06:37:27.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.800+0000 7f7ef6b2b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7ef0159e10 0x7f7ef006d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.800+0000 7f7ef6b2b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ef006d870 0x7f7ef00aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.800+0000 7f7ef6b2b640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef00b5070 con 0x7f7ef0159a30 2024-05-08T06:37:27.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.800+0000 7f7ef6b2b640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef015ffc0 con 0x7f7ef0159e10 2024-05-08T06:37:27.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.800+0000 7f7ef6b2b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef01602a0 con 0x7f7ef006d870 2024-05-08T06:37:27.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.800+0000 7f7ee7fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7ef0159e10 0x7f7ef006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.800+0000 7f7ee7fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7ef0159e10 0x7f7ef006d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53692/0 (socket says 172.21.15.163:53692) 2024-05-08T06:37:27.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.800+0000 7f7ef48c7640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f7ef0159a30 0x7f7ef00b4960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33868/0 (socket says 172.21.15.163:33868) 2024-05-08T06:37:27.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.800+0000 7f7ee7fff640 1 -- 172.21.15.163:0/3180408438 learned_addr learned my addr 172.21.15.163:0/3180408438 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:27.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.800+0000 7f7ee7fff640 1 -- 172.21.15.163:0/3180408438 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ef006d870 msgr2=0x7f7ef00aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:27.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.800+0000 7f7ee7fff640 1 --2- 172.21.15.163:0/3180408438 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ef006d870 0x7f7ef00aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.800+0000 7f7ee7fff640 1 -- 172.21.15.163:0/3180408438 >> v1:172.21.15.163:6789/0 conn(0x7f7ef0159a30 legacy=0x7f7ef00b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.800+0000 7f7ee7fff640 1 -- 172.21.15.163:0/3180408438 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef0160580 con 0x7f7ef0159e10 2024-05-08T06:37:27.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.800+0000 7f7ee77fe640 1 -- 172.21.15.163:0/3180408438 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3383743632 0 0) 0x7f7ef00b5070 con 0x7f7ef0159a30 2024-05-08T06:37:27.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.801+0000 7f7ee7fff640 1 --2- 172.21.15.163:0/3180408438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0159e10 0x7f7ef006d330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009870 tx=0x7f7ee002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf63239a2de37dbb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:27.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.801+0000 7f7ee77fe640 1 -- 172.21.15.163:0/3180408438 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0037070 con 0x7f7ef0159e10 2024-05-08T06:37:27.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.801+0000 7f7ee77fe640 1 -- 172.21.15.163:0/3180408438 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee0032440 con 0x7f7ef0159e10 2024-05-08T06:37:27.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.801+0000 7f7ee77fe640 1 -- 172.21.15.163:0/3180408438 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0032810 con 0x7f7ef0159e10 2024-05-08T06:37:27.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.802+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/3180408438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0159e10 msgr2=0x7f7ef006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.802+0000 7f7ef6b2b640 1 --2- 172.21.15.163:0/3180408438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0159e10 0x7f7ef006d330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009870 tx=0x7f7ee002a130 comp rx=0 tx=0).stop 2024-05-08T06:37:27.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.802+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/3180408438 shutdown_connections 2024-05-08T06:37:27.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.802+0000 7f7ef6b2b640 1 --2- 172.21.15.163:0/3180408438 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ef006d870 0x7f7ef00aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.802+0000 7f7ef6b2b640 1 --2- 172.21.15.163:0/3180408438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0159e10 0x7f7ef006d330 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.802+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/3180408438 >> 172.21.15.163:0/3180408438 conn(0x7f7ef0072bd0 msgr2=0x7f7ef0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:27.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.802+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/3180408438 shutdown_connections 2024-05-08T06:37:27.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.802+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/3180408438 wait complete. 2024-05-08T06:37:27.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.803+0000 7f7ef6b2b640 1 Processor -- start 2024-05-08T06:37:27.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.803+0000 7f7ef6b2b640 1 -- start start 2024-05-08T06:37:27.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ef6b2b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ef006d870 0x7f7ef0146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ef6b2b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0159a30 0x7f7ef0144400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ef6b2b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ef0159e10 0x7f7ef0143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ef6b2b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef015fd60 con 0x7f7ef0159a30 2024-05-08T06:37:27.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ef6b2b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef015fbe0 con 0x7f7ef0159e10 2024-05-08T06:37:27.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ef6b2b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef00af130 con 0x7f7ef006d870 2024-05-08T06:37:27.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0159a30 0x7f7ef0144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0159a30 0x7f7ef0144400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53696/0 (socket says 172.21.15.163:53696) 2024-05-08T06:37:27.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ee7fff640 1 -- 172.21.15.163:0/2662829024 learned_addr learned my addr 172.21.15.163:0/2662829024 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:27.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ef48c7640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ef006d870 0x7f7ef0146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ef50c8640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ef0159e10 0x7f7ef0143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ee7fff640 1 -- 172.21.15.163:0/2662829024 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ef006d870 msgr2=0x7f7ef0146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ee7fff640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ef006d870 0x7f7ef0146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ee7fff640 1 -- 172.21.15.163:0/2662829024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ef0159e10 msgr2=0x7f7ef0143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:27.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ee7fff640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ef0159e10 0x7f7ef0143b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:27.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ee7fff640 1 -- 172.21.15.163:0/2662829024 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef0144b30 con 0x7f7ef0159a30 2024-05-08T06:37:27.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ef48c7640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ef006d870 0x7f7ef0146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:27.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ef50c8640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ef0159e10 0x7f7ef0143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:27.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ee7fff640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0159a30 0x7f7ef0144400 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0000c00 tx=0x7f7ee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:27.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.804+0000 7f7ee57fa640 1 -- 172.21.15.163:0/2662829024 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0041070 con 0x7f7ef0159a30 2024-05-08T06:37:27.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.805+0000 7f7ee57fa640 1 -- 172.21.15.163:0/2662829024 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee003c870 con 0x7f7ef0159a30 2024-05-08T06:37:27.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.805+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/2662829024 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef0144e10 con 0x7f7ef0159a30 2024-05-08T06:37:27.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.805+0000 7f7ee57fa640 1 -- 172.21.15.163:0/2662829024 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0037040 con 0x7f7ef0159a30 2024-05-08T06:37:27.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.805+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/2662829024 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef01453d0 con 0x7f7ef0159a30 2024-05-08T06:37:27.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.806+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/2662829024 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef00cffc0 con 0x7f7ef0159a30 2024-05-08T06:37:27.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.806+0000 7f7ee57fa640 1 -- 172.21.15.163:0/2662829024 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7ee0044070 con 0x7f7ef0159a30 2024-05-08T06:37:27.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.807+0000 7f7ee57fa640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7edc0762d0 0x7f7edc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:27.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.807+0000 7f7ee57fa640 1 -- 172.21.15.163:0/2662829024 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f7ee00b8210 con 0x7f7ef0159a30 2024-05-08T06:37:27.807 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.807+0000 7f7ef48c7640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7edc0762d0 0x7f7edc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:27.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.807+0000 7f7ef48c7640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7edc0762d0 0x7f7edc078790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0012e0 tx=0x7f7eec007f50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:27.810 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:27.811+0000 7f7ee57fa640 1 -- 172.21.15.163:0/2662829024 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7ef00cffc0 con 0x7f7ef0159a30 2024-05-08T06:37:27.823 INFO:tasks.ceph.mds.a.smithi163.stderr:2024-05-08T06:37:27.824+0000 7fdc28b40640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T06:37:28.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28.030+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/2662829024 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7ef006a4b0 con 0x7f7ef0159a30 2024-05-08T06:37:28.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28.030+0000 7f7ee57fa640 1 -- 172.21.15.163:0/2662829024 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7f7ee0081b70 con 0x7f7ef0159a30 2024-05-08T06:37:28.030 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T06:37:28.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28.032+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/2662829024 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7edc0762d0 msgr2=0x7f7edc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:28.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28.032+0000 7f7ef6b2b640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7edc0762d0 0x7f7edc078790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0012e0 tx=0x7f7eec007f50 comp rx=0 tx=0).stop 2024-05-08T06:37:28.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28.032+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/2662829024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0159a30 msgr2=0x7f7ef0144400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:28.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28.032+0000 7f7ef6b2b640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0159a30 0x7f7ef0144400 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0000c00 tx=0x7f7ee00046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:28.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28.032+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/2662829024 shutdown_connections 2024-05-08T06:37:28.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28.032+0000 7f7ef6b2b640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7edc0762d0 0x7f7edc078790 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:28.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28.032+0000 7f7ef6b2b640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ef0159e10 0x7f7ef0143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:28.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28.032+0000 7f7ef6b2b640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ef0159a30 0x7f7ef0144400 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:28.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28.032+0000 7f7ef6b2b640 1 --2- 172.21.15.163:0/2662829024 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ef006d870 0x7f7ef0146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:28.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28.032+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/2662829024 >> 172.21.15.163:0/2662829024 conn(0x7f7ef0072bd0 msgr2=0x7f7ef00cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:28.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28.033+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/2662829024 shutdown_connections 2024-05-08T06:37:28.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28.033+0000 7f7ef6b2b640 1 -- 172.21.15.163:0/2662829024 wait complete. 2024-05-08T06:37:28.043 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T06:37:28.043 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T06:37:28.043 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2024-05-08T06:37:28.043 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T06:37:28.043 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T06:37:28.043 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T06:37:28.043 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T06:37:28.043 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T06:37:28.043 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T06:37:28.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:28.044 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2024-05-08T06:37:28.060 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T06:37:28.060 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T06:37:28.060 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T06:37:28.060 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:28.060 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2024-05-08T06:37:28.060 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:28.060 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:37:28.060 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:28.060 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2024-05-08T06:37:28.060 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:37:28.060 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:28.061 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2024-05-08T06:37:28.061 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:37:28.061 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:28.061 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T06:37:28.061 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:28.061 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-08T06:37:28.061 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42738sec preferred_lft 42738sec 2024-05-08T06:37:28.061 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2024-05-08T06:37:28.061 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:28.061 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T06:37:28.061 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 12:81:51:29:75:57 brd ff:ff:ff:ff:ff:ff 2024-05-08T06:37:28.061 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T06:37:28.061 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:28.061 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::54c1:2dff:fec0:8aa2/64 scope link 2024-05-08T06:37:28.062 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:28.062 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T06:37:28.062 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 5e:ea:b2:d9:8a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T06:37:28.062 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::5cea:b2ff:fed9:8a9c/64 scope link 2024-05-08T06:37:28.062 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:28.062 INFO:teuthology.orchestra.run.smithi163.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T06:37:28.062 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 12:81:51:29:75:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T06:37:28.062 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::1081:51ff:fe29:7557/64 scope link 2024-05-08T06:37:28.062 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:37:28.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:28.063 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T06:37:28.123 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-05-08T06:37:28.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:28.125 DEBUG:teuthology.orchestra.run.smithi163:> route 2024-05-08T06:37:28.181 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2024-05-08T06:37:28.181 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T06:37:28.181 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T06:37:28.181 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T06:37:28.181 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T06:37:28.182 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:37:28.182 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-05-08T06:37:28.182 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T06:37:28.182 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T06:37:28.182 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T06:37:28.182 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-05-08T06:37:28.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:37:28.304 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:28.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:28.310 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-05-08T06:37:28.325 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T06:37:28.325 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T06:37:28.325 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T06:37:28.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:28.326 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2024-05-08T06:37:28.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T06:37:28.391 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T06:37:28.449 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T06:37:28.450 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T06:37:28.450 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T06:37:28.504 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-08T06:37:28.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:28.505 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T06:37:28.588 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T06:37:28.588 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T06:37:28.588 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-05-08T06:37:28.597 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T06:37:28.597 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T06:37:28.597 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-05-08T06:37:28.598 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.192:3300,172.21.15.192:3301" 2024-05-08T06:37:28.598 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T06:37:28.609 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-08T06:37:28.609 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-08T06:37:28.609 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-05-08T06:37:28.609 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.192:3300,172.21.15.192:3301" 2024-05-08T06:37:28.609 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T06:37:28.609 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.192:6789,172.21.15.192:6790" 2024-05-08T06:37:28.609 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.192:6789,172.21.15.192:6790:/ 2024-05-08T06:37:28.610 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e639506e-cf71-49c6-ae4d-5af4601c36e0" will pass to kernel 2024-05-08T06:37:28.610 INFO:teuthology.orchestra.run.smithi163.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T06:37:28.610 INFO:teuthology.orchestra.run.smithi163.stdout: You just mounted a file system that supports labels which does not 2024-05-08T06:37:28.610 INFO:teuthology.orchestra.run.smithi163.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T06:37:28.610 INFO:teuthology.orchestra.run.smithi163.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T06:37:28.610 INFO:teuthology.orchestra.run.smithi163.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T06:37:28.612 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T06:37:28.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:28.612 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T06:37:28.678 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:37:28.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:37:28.772 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:28.778 INFO:teuthology.orchestra.run.smithi163.stdout:client4858 2024-05-08T06:37:28.779 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:37:28.779 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:28.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:28.795 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir mydir) 2024-05-08T06:37:28.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:28.855 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch mydir/alpha) 2024-05-08T06:37:28.913 DEBUG:teuthology.orchestra.run.smithi163:> stdin-killer -- python3 -c ' 2024-05-08T06:37:28.913 DEBUG:teuthology.orchestra.run.smithi163:> import os 2024-05-08T06:37:28.913 DEBUG:teuthology.orchestra.run.smithi163:> import stat 2024-05-08T06:37:28.913 DEBUG:teuthology.orchestra.run.smithi163:> 2024-05-08T06:37:28.913 DEBUG:teuthology.orchestra.run.smithi163:> print(os.stat("/home/ubuntu/cephtest/mnt.0/mydir").st_ino) 2024-05-08T06:37:28.913 DEBUG:teuthology.orchestra.run.smithi163:> ' 2024-05-08T06:37:29.019 INFO:teuthology.orchestra.run.smithi163.stdout:1099511627776 2024-05-08T06:37:29.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:29.028 DEBUG:teuthology.orchestra.run.smithi163:> stdin-killer -- python3 -c ' 2024-05-08T06:37:29.028 DEBUG:teuthology.orchestra.run.smithi163:> import os 2024-05-08T06:37:29.028 DEBUG:teuthology.orchestra.run.smithi163:> import stat 2024-05-08T06:37:29.029 DEBUG:teuthology.orchestra.run.smithi163:> 2024-05-08T06:37:29.029 DEBUG:teuthology.orchestra.run.smithi163:> print(os.stat("/home/ubuntu/cephtest/mnt.0/mydir/alpha").st_ino) 2024-05-08T06:37:29.029 DEBUG:teuthology.orchestra.run.smithi163:> ' 2024-05-08T06:37:29.136 INFO:teuthology.orchestra.run.smithi163.stdout:1099511627777 2024-05-08T06:37:29.138 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:37:29.145 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:37:29.145 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:29.201 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:37:29.201 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:29.258 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T06:37:29.399 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.399+0000 7fd9feb78640 1 Processor -- start 2024-05-08T06:37:29.399 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.400+0000 7fd9feb78640 1 -- start start 2024-05-08T06:37:29.400 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.400+0000 7fd9feb78640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd9f815f350 0x7fd9f80bace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.400+0000 7fd9feb78640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd9f80d2250 0x7fd9f80d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.400+0000 7fd9feb78640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f80bb220 con 0x7fd9f80b8240 2024-05-08T06:37:29.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.400+0000 7fd9feb78640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f80d2c00 con 0x7fd9f80d2250 2024-05-08T06:37:29.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.400+0000 7fd9feb78640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f80b9670 con 0x7fd9f815f350 2024-05-08T06:37:29.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.401+0000 7fd9fd115640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd9f80d2250 0x7fd9f80d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.401+0000 7fd9fd115640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd9f80d2250 0x7fd9f80d26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53698/0 (socket says 172.21.15.163:53698) 2024-05-08T06:37:29.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.401+0000 7fd9fd115640 1 -- 172.21.15.163:0/3838979802 learned_addr learned my addr 172.21.15.163:0/3838979802 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:29.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.401+0000 7fd9fd115640 1 -- 172.21.15.163:0/3838979802 >> v2:172.21.15.192:3300/0 conn(0x7fd9f815f350 msgr2=0x7fd9f80bace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:29.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.401+0000 7fd9fd115640 1 --2- 172.21.15.163:0/3838979802 >> v2:172.21.15.192:3300/0 conn(0x7fd9f815f350 0x7fd9f80bace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.401+0000 7fd9fd115640 1 -- 172.21.15.163:0/3838979802 >> v1:172.21.15.192:6789/0 conn(0x7fd9f80b8240 legacy=0x7fd9f80d3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:29.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.401+0000 7fd9fd115640 1 -- 172.21.15.163:0/3838979802 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f80b8a60 con 0x7fd9f80d2250 2024-05-08T06:37:29.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.402+0000 7fd9fd115640 1 --2- 172.21.15.163:0/3838979802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f80d2250 0x7fd9f80d26c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8163f80 tx=0x7fd9f002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc21cb07acd59e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.402+0000 7fd9f77fe640 1 -- 172.21.15.163:0/3838979802 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0036070 con 0x7fd9f80d2250 2024-05-08T06:37:29.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.402+0000 7fd9f77fe640 1 -- 172.21.15.163:0/3838979802 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f0031600 con 0x7fd9f80d2250 2024-05-08T06:37:29.402 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.402+0000 7fd9f77fe640 1 -- 172.21.15.163:0/3838979802 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00319d0 con 0x7fd9f80d2250 2024-05-08T06:37:29.403 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.402+0000 7fd9feb78640 1 -- 172.21.15.163:0/3838979802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f80d2250 msgr2=0x7fd9f80d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.403 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.402+0000 7fd9feb78640 1 --2- 172.21.15.163:0/3838979802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f80d2250 0x7fd9f80d26c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8163f80 tx=0x7fd9f002a710 comp rx=0 tx=0).stop 2024-05-08T06:37:29.403 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.402+0000 7fd9feb78640 1 -- 172.21.15.163:0/3838979802 shutdown_connections 2024-05-08T06:37:29.403 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.402+0000 7fd9feb78640 1 --2- 172.21.15.163:0/3838979802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f80d2250 0x7fd9f80d26c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.403 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.402+0000 7fd9feb78640 1 --2- 172.21.15.163:0/3838979802 >> v2:172.21.15.192:3300/0 conn(0x7fd9f815f350 0x7fd9f80bace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.403 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.402+0000 7fd9feb78640 1 -- 172.21.15.163:0/3838979802 >> 172.21.15.163:0/3838979802 conn(0x7fd9f8073e70 msgr2=0x7fd9f8074280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:29.403 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.402+0000 7fd9feb78640 1 -- 172.21.15.163:0/3838979802 shutdown_connections 2024-05-08T06:37:29.403 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.402+0000 7fd9feb78640 1 -- 172.21.15.163:0/3838979802 wait complete. 2024-05-08T06:37:29.403 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.403+0000 7fd9feb78640 1 Processor -- start 2024-05-08T06:37:29.403 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.404+0000 7fd9feb78640 1 -- start start 2024-05-08T06:37:29.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.404+0000 7fd9feb78640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9f80b8240 0x7fd9f81603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.404+0000 7fd9feb78640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9f80d2250 0x7fd9f81f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.404+0000 7fd9feb78640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f815f350 0x7fd9f81f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.404+0000 7fd9feb78640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f80b9300 con 0x7fd9f815f350 2024-05-08T06:37:29.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.404+0000 7fd9feb78640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f80b9020 con 0x7fd9f80b8240 2024-05-08T06:37:29.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.404+0000 7fd9feb78640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f815f730 con 0x7fd9f80d2250 2024-05-08T06:37:29.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.404+0000 7fd9fd115640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f815f350 0x7fd9f81f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9fd115640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f815f350 0x7fd9f81f8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53714/0 (socket says 172.21.15.163:53714) 2024-05-08T06:37:29.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9fd115640 1 -- 172.21.15.163:0/4082790786 learned_addr learned my addr 172.21.15.163:0/4082790786 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:29.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9fc914640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9f80b8240 0x7fd9f81603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9fd115640 1 -- 172.21.15.163:0/4082790786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9f80d2250 msgr2=0x7fd9f81f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9f7fff640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9f80d2250 0x7fd9f81f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9fd115640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9f80d2250 0x7fd9f81f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9fd115640 1 -- 172.21.15.163:0/4082790786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9f80b8240 msgr2=0x7fd9f81603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9fd115640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9f80b8240 0x7fd9f81603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9fd115640 1 -- 172.21.15.163:0/4082790786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f815bd20 con 0x7fd9f815f350 2024-05-08T06:37:29.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9fc914640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9f80b8240 0x7fd9f81603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:29.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9f7fff640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9f80d2250 0x7fd9f81f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:29.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9fd115640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f815f350 0x7fd9f81f8100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0034040 tx=0x7fd9f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9f57fa640 1 -- 172.21.15.163:0/4082790786 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0041020 con 0x7fd9f815f350 2024-05-08T06:37:29.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9f57fa640 1 -- 172.21.15.163:0/4082790786 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f00094d0 con 0x7fd9f815f350 2024-05-08T06:37:29.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9feb78640 1 -- 172.21.15.163:0/4082790786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f8160a60 con 0x7fd9f815f350 2024-05-08T06:37:29.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9f57fa640 1 -- 172.21.15.163:0/4082790786 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0036040 con 0x7fd9f815f350 2024-05-08T06:37:29.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.405+0000 7fd9feb78640 1 -- 172.21.15.163:0/4082790786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f8161020 con 0x7fd9f815f350 2024-05-08T06:37:29.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.406+0000 7fd9feb78640 1 -- 172.21.15.163:0/4082790786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f806a4b0 con 0x7fd9f815f350 2024-05-08T06:37:29.408 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.407+0000 7fd9f57fa640 1 -- 172.21.15.163:0/4082790786 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd9f0044020 con 0x7fd9f815f350 2024-05-08T06:37:29.408 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.407+0000 7fd9f57fa640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd9e40762d0 0x7fd9e4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.408 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.407+0000 7fd9f57fa640 1 -- 172.21.15.163:0/4082790786 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fd9f00b8320 con 0x7fd9f815f350 2024-05-08T06:37:29.408 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.407+0000 7fd9fc914640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd9e40762d0 0x7fd9e4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.408 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.408+0000 7fd9fc914640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd9e40762d0 0x7fd9e4078790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800a870 tx=0x7fd9e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.410 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.412+0000 7fd9f57fa640 1 -- 172.21.15.163:0/4082790786 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd9f004b030 con 0x7fd9f815f350 2024-05-08T06:37:29.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.603+0000 7fd9feb78640 1 -- 172.21.15.163:0/4082790786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd9f80b5ec0 con 0x7fd9f815f350 2024-05-08T06:37:29.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.603+0000 7fd9f57fa640 1 -- 172.21.15.163:0/4082790786 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v31) ==== 80+0+0 (secure 0 0 0) 0x7fd9f0081dd0 con 0x7fd9f815f350 2024-05-08T06:37:29.602 INFO:teuthology.orchestra.run.smithi163.stderr:listed 0 entries 2024-05-08T06:37:29.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.605+0000 7fd9feb78640 1 -- 172.21.15.163:0/4082790786 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd9e40762d0 msgr2=0x7fd9e4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.605+0000 7fd9feb78640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd9e40762d0 0x7fd9e4078790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800a870 tx=0x7fd9e8008040 comp rx=0 tx=0).stop 2024-05-08T06:37:29.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.605+0000 7fd9feb78640 1 -- 172.21.15.163:0/4082790786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f815f350 msgr2=0x7fd9f81f8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.605+0000 7fd9feb78640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f815f350 0x7fd9f81f8100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0034040 tx=0x7fd9f0004a40 comp rx=0 tx=0).stop 2024-05-08T06:37:29.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.605+0000 7fd9feb78640 1 -- 172.21.15.163:0/4082790786 shutdown_connections 2024-05-08T06:37:29.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.605+0000 7fd9feb78640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd9e40762d0 0x7fd9e4078790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.605+0000 7fd9feb78640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f815f350 0x7fd9f81f8100 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.605+0000 7fd9feb78640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9f80d2250 0x7fd9f81f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.605+0000 7fd9feb78640 1 --2- 172.21.15.163:0/4082790786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9f80b8240 0x7fd9f81603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.605+0000 7fd9feb78640 1 -- 172.21.15.163:0/4082790786 >> 172.21.15.163:0/4082790786 conn(0x7fd9f8073e70 msgr2=0x7fd9f80d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:29.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.605+0000 7fd9feb78640 1 -- 172.21.15.163:0/4082790786 shutdown_connections 2024-05-08T06:37:29.606 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.605+0000 7fd9feb78640 1 -- 172.21.15.163:0/4082790786 wait complete. 2024-05-08T06:37:29.616 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T06:37:29.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:29.616 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T06:37:29.657 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:37:29.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:29.658 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T06:37:29.713 DEBUG:teuthology.orchestra.run.smithi163:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2024-05-08T06:37:29.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.797+0000 7f4c52e92f40 1 Processor -- start 2024-05-08T06:37:29.797 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.798+0000 7f4c52e92f40 1 -- start start 2024-05-08T06:37:29.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.798+0000 7f4c52e92f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c3c0028f0 0x5646a7134ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.798+0000 7f4c52e92f40 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5646a70a6520 0x5646a71c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.798+0000 7f4c52e92f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646a71c1ae0 0x5646a70a2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.798+0000 7f4c52e92f40 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5646a7002f50 con 0x5646a70a6520 2024-05-08T06:37:29.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.798+0000 7f4c52e92f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5646a7006f90 con 0x5646a71c1ae0 2024-05-08T06:37:29.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.798+0000 7f4c52e92f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5646a70a38f0 con 0x7f4c3c0028f0 2024-05-08T06:37:29.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.798+0000 7f4c4818d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5646a70a6520 0x5646a71c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.798+0000 7f4c4818d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5646a70a6520 0x5646a71c15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38522/0 (socket says 172.21.15.163:38522) 2024-05-08T06:37:29.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.798+0000 7f4c4818d640 1 -- 172.21.15.163:0/1557590277 learned_addr learned my addr 172.21.15.163:0/1557590277 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:29.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.798+0000 7f4c5118f640 1 --2- 172.21.15.163:0/1557590277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646a71c1ae0 0x5646a70a2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.798+0000 7f4c5098e640 1 --2- 172.21.15.163:0/1557590277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c3c0028f0 0x5646a7134ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c4818d640 1 -- 172.21.15.163:0/1557590277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c3c0028f0 msgr2=0x5646a7134ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:29.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c4818d640 1 --2- 172.21.15.163:0/1557590277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c3c0028f0 0x5646a7134ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c4818d640 1 -- 172.21.15.163:0/1557590277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646a71c1ae0 msgr2=0x5646a70a2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:29.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c4818d640 1 --2- 172.21.15.163:0/1557590277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646a71c1ae0 0x5646a70a2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.802 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c4818d640 1 -- 172.21.15.163:0/1557590277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a7087590 con 0x5646a70a6520 2024-05-08T06:37:29.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c5098e640 1 --2- 172.21.15.163:0/1557590277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c3c0028f0 0x5646a7134ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:29.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c4818d640 1 --2- 172.21.15.163:0/1557590277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5646a70a6520 0x5646a71c15a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4c38057040 tx=0x7f4c3804f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81afe37f7e4451ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c4b7fe640 1 -- 172.21.15.163:0/1557590277 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38069070 con 0x5646a70a6520 2024-05-08T06:37:29.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c4b7fe640 1 -- 172.21.15.163:0/1557590277 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c380014b0 con 0x5646a70a6520 2024-05-08T06:37:29.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c4b7fe640 1 -- 172.21.15.163:0/1557590277 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38064410 con 0x5646a70a6520 2024-05-08T06:37:29.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c52e92f40 1 -- 172.21.15.163:0/1557590277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5646a70a6520 msgr2=0x5646a71c15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c52e92f40 1 --2- 172.21.15.163:0/1557590277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5646a70a6520 0x5646a71c15a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4c38057040 tx=0x7f4c3804f000 comp rx=0 tx=0).stop 2024-05-08T06:37:29.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c52e92f40 1 -- 172.21.15.163:0/1557590277 shutdown_connections 2024-05-08T06:37:29.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c52e92f40 1 --2- 172.21.15.163:0/1557590277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646a71c1ae0 0x5646a70a2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c52e92f40 1 --2- 172.21.15.163:0/1557590277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5646a70a6520 0x5646a71c15a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.803 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c52e92f40 1 --2- 172.21.15.163:0/1557590277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c3c0028f0 0x5646a7134ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c52e92f40 1 -- 172.21.15.163:0/1557590277 >> 172.21.15.163:0/1557590277 conn(0x5646a70a5f10 msgr2=0x5646a7135b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c52e92f40 1 -- 172.21.15.163:0/1557590277 shutdown_connections 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.799+0000 7f4c52e92f40 1 -- 172.21.15.163:0/1557590277 wait complete. 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.800+0000 7f4c52e92f40 1 Processor -- start 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.800+0000 7f4c52e92f40 1 -- start start 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c52e92f40 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5646a71c1ae0 0x5646a71c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c52e92f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c3c0028f0 0x5646a7131170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c52e92f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5646a712d370 0x5646a7134ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c52e92f40 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5646a70a1bc0 con 0x5646a71c1ae0 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c52e92f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5646a70a18e0 con 0x7f4c3c0028f0 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c52e92f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5646a712df30 con 0x5646a712d370 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c5098e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5646a71c1ae0 0x5646a71c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c5098e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5646a71c1ae0 0x5646a71c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38526/0 (socket says 172.21.15.163:38526) 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c5098e640 1 -- 172.21.15.163:0/3268387532 learned_addr learned my addr 172.21.15.163:0/3268387532 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c4818d640 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c3c0028f0 0x5646a7131170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.804 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c5118f640 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5646a712d370 0x5646a7134ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c5098e640 1 -- 172.21.15.163:0/3268387532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5646a712d370 msgr2=0x5646a7134ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c5098e640 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5646a712d370 0x5646a7134ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c5098e640 1 -- 172.21.15.163:0/3268387532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c3c0028f0 msgr2=0x5646a7131170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c5098e640 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c3c0028f0 0x5646a7131170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c5098e640 1 -- 172.21.15.163:0/3268387532 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a71c9fe0 con 0x5646a71c1ae0 2024-05-08T06:37:29.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c5118f640 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5646a712d370 0x5646a7134ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:29.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c5098e640 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5646a71c1ae0 0x5646a71c9aa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4c40005630 tx=0x7f4c40009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c4a7fc640 1 -- 172.21.15.163:0/3268387532 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4001a020 con 0x5646a71c1ae0 2024-05-08T06:37:29.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c4a7fc640 1 -- 172.21.15.163:0/3268387532 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c40016440 con 0x5646a71c1ae0 2024-05-08T06:37:29.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c52e92f40 1 -- 172.21.15.163:0/3268387532 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5646a71ca280 con 0x5646a71c1ae0 2024-05-08T06:37:29.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c4a7fc640 1 -- 172.21.15.163:0/3268387532 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40016810 con 0x5646a71c1ae0 2024-05-08T06:37:29.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.801+0000 7f4c52e92f40 1 -- 172.21.15.163:0/3268387532 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a71ca840 con 0x5646a71c1ae0 2024-05-08T06:37:29.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:37:29.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.803+0000 7f4c4a7fc640 1 -- 172.21.15.163:0/3268387532 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4c4001d020 con 0x5646a71c1ae0 2024-05-08T06:37:29.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.803+0000 7f4c4a7fc640 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4c30074cb0 0x7f4c30077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.803+0000 7f4c4818d640 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4c30074cb0 0x7f4c30077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.803+0000 7f4c4a7fc640 1 -- 172.21.15.163:0/3268387532 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f4c400965d0 con 0x5646a71c1ae0 2024-05-08T06:37:29.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c52e92f40 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x5646a71cb940 0x5646a71cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c52e92f40 1 -- 172.21.15.163:0/3268387532 --> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646a71cc260 con 0x5646a71cb940 2024-05-08T06:37:29.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c5118f640 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x5646a71cb940 0x5646a71cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c4818d640 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4c30074cb0 0x7f4c30077170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4c38001860 tx=0x7f4c38051000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c5118f640 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x5646a71cb940 0x5646a71cbd20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c5118f640 1 -- 172.21.15.163:0/3268387532 <== osd.3 v2:172.21.15.163:6816/127592521 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 164+0+0 (crc 0 0 0) 0x7f4c4c0112c0 con 0x5646a71cb940 2024-05-08T06:37:29.809 INFO:teuthology.orchestra.run.smithi163.stderr:error getting omap key set cephfs_metadata/10000000000.00000000: (2) No such file or directory 2024-05-08T06:37:29.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c52e92f40 1 -- 172.21.15.163:0/3268387532 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x5646a71cb940 msgr2=0x5646a71cbd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c52e92f40 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x5646a71cb940 0x5646a71cbd20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c52e92f40 1 -- 172.21.15.163:0/3268387532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4c30074cb0 msgr2=0x7f4c30077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c52e92f40 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4c30074cb0 0x7f4c30077170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4c38001860 tx=0x7f4c38051000 comp rx=0 tx=0).stop 2024-05-08T06:37:29.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c52e92f40 1 -- 172.21.15.163:0/3268387532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5646a71c1ae0 msgr2=0x5646a71c9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c52e92f40 1 --2- 172.21.15.163:0/3268387532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5646a71c1ae0 0x5646a71c9aa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4c40005630 tx=0x7f4c40009ef0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c5118f640 1 -- 172.21.15.163:0/3268387532 reap_dead start 2024-05-08T06:37:29.810 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c52e92f40 1 -- 172.21.15.163:0/3268387532 shutdown_connections 2024-05-08T06:37:29.810 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.804+0000 7f4c52e92f40 1 -- 172.21.15.163:0/3268387532 >> 172.21.15.163:0/3268387532 conn(0x5646a70a5f10 msgr2=0x5646a70a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:29.810 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.805+0000 7f4c52e92f40 1 -- 172.21.15.163:0/3268387532 shutdown_connections 2024-05-08T06:37:29.810 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.805+0000 7f4c52e92f40 1 -- 172.21.15.163:0/3268387532 wait complete. 2024-05-08T06:37:29.810 ERROR:tasks.cephfs.filesystem:Command failed on smithi163 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000' 2024-05-08T06:37:29.810 DEBUG:teuthology.orchestra.run.smithi163:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-08T06:37:29.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.894+0000 7fd3c473ef40 1 Processor -- start 2024-05-08T06:37:29.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.894+0000 7fd3c473ef40 1 -- start start 2024-05-08T06:37:29.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.894+0000 7fd3c473ef40 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c397bd81a0 0x55c397c76960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.894+0000 7fd3c473ef40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c397c807d0 0x55c397c81f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.894+0000 7fd3c473ef40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c397c80bb0 0x55c397bd4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.894+0000 7fd3c473ef40 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c397bd3710 con 0x55c397bd81a0 2024-05-08T06:37:29.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.894+0000 7fd3c473ef40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c397b50f50 con 0x55c397c80bb0 2024-05-08T06:37:29.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.894+0000 7fd3c473ef40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55c397bf18f0 con 0x55c397c807d0 2024-05-08T06:37:29.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.894+0000 7fd3c2238640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c397bd81a0 0x55c397c76960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c2238640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c397bd81a0 0x55c397c76960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38558/0 (socket says 172.21.15.163:38558) 2024-05-08T06:37:29.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c2238640 1 -- 172.21.15.163:0/2003185799 learned_addr learned my addr 172.21.15.163:0/2003185799 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:29.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c2238640 1 -- 172.21.15.163:0/2003185799 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c397c807d0 msgr2=0x55c397c81f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:29.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c2238640 1 --2- 172.21.15.163:0/2003185799 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c397c807d0 0x55c397c81f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c2238640 1 -- 172.21.15.163:0/2003185799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c397c80bb0 msgr2=0x55c397bd4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:29.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c2238640 1 --2- 172.21.15.163:0/2003185799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c397c80bb0 0x55c397bd4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c2238640 1 -- 172.21.15.163:0/2003185799 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c397c7dee0 con 0x55c397bd81a0 2024-05-08T06:37:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c2238640 1 --2- 172.21.15.163:0/2003185799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c397bd81a0 0x55c397c76960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40095d0 tx=0x7fd3b4009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7a535628021de4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c1236640 1 -- 172.21.15.163:0/2003185799 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4006680 con 0x55c397bd81a0 2024-05-08T06:37:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c1236640 1 -- 172.21.15.163:0/2003185799 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b4006820 con 0x55c397bd81a0 2024-05-08T06:37:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c1236640 1 -- 172.21.15.163:0/2003185799 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4006c10 con 0x55c397bd81a0 2024-05-08T06:37:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c473ef40 1 -- 172.21.15.163:0/2003185799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c397bd81a0 msgr2=0x55c397c76960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c473ef40 1 --2- 172.21.15.163:0/2003185799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c397bd81a0 0x55c397c76960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40095d0 tx=0x7fd3b4009a90 comp rx=0 tx=0).stop 2024-05-08T06:37:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c473ef40 1 -- 172.21.15.163:0/2003185799 shutdown_connections 2024-05-08T06:37:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c473ef40 1 --2- 172.21.15.163:0/2003185799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c397c80bb0 0x55c397bd4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c473ef40 1 --2- 172.21.15.163:0/2003185799 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c397c807d0 0x55c397c81f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.895+0000 7fd3c473ef40 1 --2- 172.21.15.163:0/2003185799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c397bd81a0 0x55c397c76960 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.896+0000 7fd3c473ef40 1 -- 172.21.15.163:0/2003185799 >> 172.21.15.163:0/2003185799 conn(0x55c397bf3f10 msgr2=0x55c397c7d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.896+0000 7fd3c473ef40 1 -- 172.21.15.163:0/2003185799 shutdown_connections 2024-05-08T06:37:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.896+0000 7fd3c473ef40 1 -- 172.21.15.163:0/2003185799 wait complete. 2024-05-08T06:37:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.896+0000 7fd3c473ef40 1 Processor -- start 2024-05-08T06:37:29.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.896+0000 7fd3c473ef40 1 -- start start 2024-05-08T06:37:29.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c473ef40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c397bd81a0 0x55c397c7f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c473ef40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c397c807d0 0x55c397c77de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c473ef40 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c397c80bb0 0x55c397c74010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c473ef40 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c397beff40 con 0x55c397c80bb0 2024-05-08T06:37:29.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c473ef40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c397c84380 con 0x55c397bd81a0 2024-05-08T06:37:29.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c473ef40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55c397bf0190 con 0x55c397c807d0 2024-05-08T06:37:29.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c2a39640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c397c80bb0 0x55c397c74010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c2a39640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c397c80bb0 0x55c397c74010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38560/0 (socket says 172.21.15.163:38560) 2024-05-08T06:37:29.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c2a39640 1 -- 172.21.15.163:0/1055349840 learned_addr learned my addr 172.21.15.163:0/1055349840 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:29.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3b9a37640 1 --2- 172.21.15.163:0/1055349840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c397c807d0 0x55c397c77de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c2238640 1 --2- 172.21.15.163:0/1055349840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c397bd81a0 0x55c397c7f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c2a39640 1 -- 172.21.15.163:0/1055349840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c397c807d0 msgr2=0x55c397c77de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c2a39640 1 --2- 172.21.15.163:0/1055349840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c397c807d0 0x55c397c77de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c2a39640 1 -- 172.21.15.163:0/1055349840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c397bd81a0 msgr2=0x55c397c7f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c2a39640 1 --2- 172.21.15.163:0/1055349840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c397bd81a0 0x55c397c7f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c2a39640 1 -- 172.21.15.163:0/1055349840 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c397c6d4c0 con 0x55c397c80bb0 2024-05-08T06:37:29.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.897+0000 7fd3c2a39640 1 --2- 172.21.15.163:0/1055349840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c397c80bb0 0x55c397c74010 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00a4a0 tx=0x7fd3bc046010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.898+0000 7fd3bbfff640 1 -- 172.21.15.163:0/1055349840 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc058600 con 0x55c397c80bb0 2024-05-08T06:37:29.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.898+0000 7fd3bbfff640 1 -- 172.21.15.163:0/1055349840 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3bc067070 con 0x55c397c80bb0 2024-05-08T06:37:29.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.898+0000 7fd3bbfff640 1 -- 172.21.15.163:0/1055349840 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc062610 con 0x55c397c80bb0 2024-05-08T06:37:29.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.898+0000 7fd3c473ef40 1 -- 172.21.15.163:0/1055349840 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c397c6d760 con 0x55c397c80bb0 2024-05-08T06:37:29.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.898+0000 7fd3c473ef40 1 -- 172.21.15.163:0/1055349840 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c397c7fe40 con 0x55c397c80bb0 2024-05-08T06:37:29.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.899+0000 7fd3bbfff640 1 -- 172.21.15.163:0/1055349840 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd3bc0627b0 con 0x55c397c80bb0 2024-05-08T06:37:29.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.899+0000 7fd3bbfff640 1 --2- 172.21.15.163:0/1055349840 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd3a4074ce0 0x7fd3a40771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.900+0000 7fd3bbfff640 1 -- 172.21.15.163:0/1055349840 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fd3bc0da330 con 0x55c397c80bb0 2024-05-08T06:37:29.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.900+0000 7fd3c2238640 1 --2- 172.21.15.163:0/1055349840 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd3a4074ce0 0x7fd3a40771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.900+0000 7fd3c473ef40 1 --2- 172.21.15.163:0/1055349840 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x55c397c74550 0x55c397d1aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.900+0000 7fd3c473ef40 1 -- 172.21.15.163:0/1055349840 --> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c397c7ec30 con 0x55c397c74550 2024-05-08T06:37:29.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.900+0000 7fd3b9a37640 1 --2- 172.21.15.163:0/1055349840 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x55c397c74550 0x55c397d1aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.900+0000 7fd3c2238640 1 --2- 172.21.15.163:0/1055349840 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd3a4074ce0 0x7fd3a40771a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400afd0 tx=0x7fd3b400f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.900+0000 7fd3b9a37640 1 --2- 172.21.15.163:0/1055349840 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x55c397c74550 0x55c397d1aa40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.901+0000 7fd3b9a37640 1 -- 172.21.15.163:0/1055349840 <== osd.5 v2:172.21.15.192:6800/2911825781 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 164+0+0 (crc 0 0 0) 0x7fd3ac00e500 con 0x55c397c74550 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:error getting xattr cephfs_data/10000000001.00000000/parent: (2) No such file or directory 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.901+0000 7fd3c473ef40 1 -- 172.21.15.163:0/1055349840 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x55c397c74550 msgr2=0x55c397d1aa40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.901+0000 7fd3c473ef40 1 --2- 172.21.15.163:0/1055349840 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x55c397c74550 0x55c397d1aa40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.901+0000 7fd3c473ef40 1 -- 172.21.15.163:0/1055349840 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd3a4074ce0 msgr2=0x7fd3a40771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.901+0000 7fd3c473ef40 1 --2- 172.21.15.163:0/1055349840 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd3a4074ce0 0x7fd3a40771a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400afd0 tx=0x7fd3b400f000 comp rx=0 tx=0).stop 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.901+0000 7fd3c473ef40 1 -- 172.21.15.163:0/1055349840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c397c80bb0 msgr2=0x55c397c74010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.901+0000 7fd3c473ef40 1 --2- 172.21.15.163:0/1055349840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c397c80bb0 0x55c397c74010 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00a4a0 tx=0x7fd3bc046010 comp rx=0 tx=0).stop 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.901+0000 7fd3c2a39640 1 -- 172.21.15.163:0/1055349840 reap_dead start 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.901+0000 7fd3c473ef40 1 -- 172.21.15.163:0/1055349840 shutdown_connections 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.901+0000 7fd3c473ef40 1 -- 172.21.15.163:0/1055349840 >> 172.21.15.163:0/1055349840 conn(0x55c397bf3f10 msgr2=0x55c397c7e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.901+0000 7fd3c473ef40 1 -- 172.21.15.163:0/1055349840 shutdown_connections 2024-05-08T06:37:29.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.901+0000 7fd3c473ef40 1 -- 172.21.15.163:0/1055349840 wait complete. 2024-05-08T06:37:29.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:37:29.905 ERROR:tasks.cephfs.filesystem:Command failed on smithi163 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent' 2024-05-08T06:37:29.906 DEBUG:teuthology.orchestra.run.smithi163:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2024-05-08T06:37:29.949 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.949+0000 7fb42c962f40 1 Processor -- start 2024-05-08T06:37:29.949 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.950+0000 7fb42c962f40 1 -- start start 2024-05-08T06:37:29.953 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.950+0000 7fb42c962f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa0087ed0 0x558fa0153740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.953 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.950+0000 7fb42c962f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558fa00cc880 0x558fa0156090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.953 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.950+0000 7fb42c962f40 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558fa015cd20 0x558fa01565d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.953 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.950+0000 7fb42c962f40 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558fa0088820 con 0x558fa015cd20 2024-05-08T06:37:29.953 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.950+0000 7fb42c962f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558fa00cf6a0 con 0x558fa0087ed0 2024-05-08T06:37:29.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.950+0000 7fb42c962f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x558fa00b3570 con 0x558fa00cc880 2024-05-08T06:37:29.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.950+0000 7fb42ac5d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558fa015cd20 0x558fa01565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.950+0000 7fb42ac5d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558fa015cd20 0x558fa01565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38578/0 (socket says 172.21.15.163:38578) 2024-05-08T06:37:29.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.950+0000 7fb42ac5d640 1 -- 172.21.15.163:0/1746380353 learned_addr learned my addr 172.21.15.163:0/1746380353 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:29.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.950+0000 7fb421c5b640 1 --2- 172.21.15.163:0/1746380353 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558fa00cc880 0x558fa0156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.950+0000 7fb42a45c640 1 --2- 172.21.15.163:0/1746380353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa0087ed0 0x558fa0153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42ac5d640 1 -- 172.21.15.163:0/1746380353 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558fa00cc880 msgr2=0x558fa0156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:29.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42ac5d640 1 --2- 172.21.15.163:0/1746380353 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558fa00cc880 0x558fa0156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42ac5d640 1 -- 172.21.15.163:0/1746380353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa0087ed0 msgr2=0x558fa0153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:29.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42ac5d640 1 --2- 172.21.15.163:0/1746380353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa0087ed0 0x558fa0153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42ac5d640 1 -- 172.21.15.163:0/1746380353 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f9ffed350 con 0x558fa015cd20 2024-05-08T06:37:29.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42ac5d640 1 --2- 172.21.15.163:0/1746380353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558fa015cd20 0x558fa01565d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb42400b630 tx=0x7fb42400baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe5a8ec55d7ae62 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42945a640 1 -- 172.21.15.163:0/1746380353 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424014020 con 0x558fa015cd20 2024-05-08T06:37:29.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42945a640 1 -- 172.21.15.163:0/1746380353 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb424010820 con 0x558fa015cd20 2024-05-08T06:37:29.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42945a640 1 -- 172.21.15.163:0/1746380353 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424010c10 con 0x558fa015cd20 2024-05-08T06:37:29.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42c962f40 1 -- 172.21.15.163:0/1746380353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558fa015cd20 msgr2=0x558fa01565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42c962f40 1 --2- 172.21.15.163:0/1746380353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558fa015cd20 0x558fa01565d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb42400b630 tx=0x7fb42400baf0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42c962f40 1 -- 172.21.15.163:0/1746380353 shutdown_connections 2024-05-08T06:37:29.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42c962f40 1 --2- 172.21.15.163:0/1746380353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558fa015cd20 0x558fa01565d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42c962f40 1 --2- 172.21.15.163:0/1746380353 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558fa00cc880 0x558fa0156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42c962f40 1 --2- 172.21.15.163:0/1746380353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa0087ed0 0x558fa0153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42c962f40 1 -- 172.21.15.163:0/1746380353 >> 172.21.15.163:0/1746380353 conn(0x558fa0083150 msgr2=0x558fa0154810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:29.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42c962f40 1 -- 172.21.15.163:0/1746380353 shutdown_connections 2024-05-08T06:37:29.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.951+0000 7fb42c962f40 1 -- 172.21.15.163:0/1746380353 wait complete. 2024-05-08T06:37:29.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.952+0000 7fb42c962f40 1 Processor -- start 2024-05-08T06:37:29.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.952+0000 7fb42c962f40 1 -- start start 2024-05-08T06:37:29.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42c962f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558fa0087ed0 0x558fa0150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42c962f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa00cc880 0x558fa01f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42c962f40 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558fa015cd20 0x558fa01f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42c962f40 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558fa00835d0 con 0x558fa015cd20 2024-05-08T06:37:29.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42c962f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558fa0153ca0 con 0x558fa00cc880 2024-05-08T06:37:29.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42c962f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x558fa0083840 con 0x558fa0087ed0 2024-05-08T06:37:29.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42ac5d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558fa015cd20 0x558fa01f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42ac5d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558fa015cd20 0x558fa01f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38580/0 (socket says 172.21.15.163:38580) 2024-05-08T06:37:29.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42ac5d640 1 -- 172.21.15.163:0/316088826 learned_addr learned my addr 172.21.15.163:0/316088826 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:29.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb421c5b640 1 --2- 172.21.15.163:0/316088826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa00cc880 0x558fa01f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42a45c640 1 --2- 172.21.15.163:0/316088826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558fa0087ed0 0x558fa0150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42ac5d640 1 -- 172.21.15.163:0/316088826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558fa0087ed0 msgr2=0x558fa0150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42ac5d640 1 --2- 172.21.15.163:0/316088826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558fa0087ed0 0x558fa0150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42ac5d640 1 -- 172.21.15.163:0/316088826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa00cc880 msgr2=0x558fa01f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42ac5d640 1 --2- 172.21.15.163:0/316088826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa00cc880 0x558fa01f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42ac5d640 1 -- 172.21.15.163:0/316088826 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa0154960 con 0x558fa015cd20 2024-05-08T06:37:29.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb42ac5d640 1 --2- 172.21.15.163:0/316088826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558fa015cd20 0x558fa01f3bf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb424005780 tx=0x7fb4240033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb423fff640 1 -- 172.21.15.163:0/316088826 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424019620 con 0x558fa015cd20 2024-05-08T06:37:29.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.953+0000 7fb423fff640 1 -- 172.21.15.163:0/316088826 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4240197e0 con 0x558fa015cd20 2024-05-08T06:37:29.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.954+0000 7fb42c962f40 1 -- 172.21.15.163:0/316088826 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558fa0154c40 con 0x558fa015cd20 2024-05-08T06:37:29.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.954+0000 7fb423fff640 1 -- 172.21.15.163:0/316088826 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424019bd0 con 0x558fa015cd20 2024-05-08T06:37:29.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.954+0000 7fb42c962f40 1 -- 172.21.15.163:0/316088826 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa0152850 con 0x558fa015cd20 2024-05-08T06:37:29.960 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.955+0000 7fb423fff640 1 -- 172.21.15.163:0/316088826 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb424017020 con 0x558fa015cd20 2024-05-08T06:37:29.960 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.955+0000 7fb423fff640 1 --2- 172.21.15.163:0/316088826 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb40c074ce0 0x7fb40c0771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.960 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.956+0000 7fb42a45c640 1 --2- 172.21.15.163:0/316088826 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb40c074ce0 0x7fb40c0771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.960 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.956+0000 7fb423fff640 1 -- 172.21.15.163:0/316088826 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fb424098400 con 0x558fa015cd20 2024-05-08T06:37:29.960 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.956+0000 7fb42c962f40 1 --2- 172.21.15.163:0/316088826 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x558fa0150f50 0x558fa01fb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:29.960 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.956+0000 7fb42c962f40 1 -- 172.21.15.163:0/316088826 --> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] -- osd_op(unknown.0.0:1 4.1f 4:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558fa014efc0 con 0x558fa0150f50 2024-05-08T06:37:29.960 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.956+0000 7fb42a45c640 1 --2- 172.21.15.163:0/316088826 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb40c074ce0 0x7fb40c0771a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb41c057000 tx=0x7fb41c061040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.956+0000 7fb421c5b640 1 --2- 172.21.15.163:0/316088826 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x558fa0150f50 0x558fa01fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:29.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.956+0000 7fb421c5b640 1 --2- 172.21.15.163:0/316088826 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x558fa0150f50 0x558fa01fb980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:29.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.957+0000 7fb421c5b640 1 -- 172.21.15.163:0/316088826 <== osd.6 v2:172.21.15.192:6816/1074108675 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) ==== 154+0+5 (crc 0 0 0) 0x7fb414009970 con 0x558fa0150f50 2024-05-08T06:37:29.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.957+0000 7fb42c962f40 1 -- 172.21.15.163:0/316088826 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x558fa0150f50 msgr2=0x558fa01fb980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.957+0000 7fb42c962f40 1 --2- 172.21.15.163:0/316088826 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x558fa0150f50 0x558fa01fb980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.957+0000 7fb42c962f40 1 -- 172.21.15.163:0/316088826 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb40c074ce0 msgr2=0x7fb40c0771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.957+0000 7fb42c962f40 1 --2- 172.21.15.163:0/316088826 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fb40c074ce0 0x7fb40c0771a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb41c057000 tx=0x7fb41c061040 comp rx=0 tx=0).stop 2024-05-08T06:37:29.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.957+0000 7fb42c962f40 1 -- 172.21.15.163:0/316088826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558fa015cd20 msgr2=0x558fa01f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:29.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.957+0000 7fb42c962f40 1 --2- 172.21.15.163:0/316088826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558fa015cd20 0x558fa01f3bf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb424005780 tx=0x7fb4240033a0 comp rx=0 tx=0).stop 2024-05-08T06:37:29.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.957+0000 7fb42ac5d640 1 -- 172.21.15.163:0/316088826 reap_dead start 2024-05-08T06:37:29.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.957+0000 7fb42c962f40 1 -- 172.21.15.163:0/316088826 shutdown_connections 2024-05-08T06:37:29.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.957+0000 7fb42c962f40 1 -- 172.21.15.163:0/316088826 >> 172.21.15.163:0/316088826 conn(0x558fa0083150 msgr2=0x558fa00b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:29.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.957+0000 7fb42c962f40 1 -- 172.21.15.163:0/316088826 shutdown_connections 2024-05-08T06:37:29.961 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:29.957+0000 7fb42c962f40 1 -- 172.21.15.163:0/316088826 wait complete. 2024-05-08T06:37:29.962 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:30.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.103+0000 7f4a53167640 1 Processor -- start 2024-05-08T06:37:30.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.103+0000 7f4a53167640 1 -- start start 2024-05-08T06:37:30.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.104+0000 7f4a53167640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c15d600 0x7f4a4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.104+0000 7f4a53167640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4a4c06f150 0x7f4a4c15e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.104+0000 7f4a53167640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a4c06f6c0 con 0x7f4a4c06d870 2024-05-08T06:37:30.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.104+0000 7f4a53167640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4a4c06dcb0 con 0x7f4a4c06f150 2024-05-08T06:37:30.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.104+0000 7f4a53167640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4c15de20 con 0x7f4a4c15d600 2024-05-08T06:37:30.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.104+0000 7f4a4bfff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4a4c06d870 0x7f4a4c06ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38354/0 (socket says 172.21.15.163:38354) 2024-05-08T06:37:30.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.104+0000 7f4a4bfff640 1 -- 172.21.15.163:0/126628914 learned_addr learned my addr 172.21.15.163:0/126628914 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:30.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.104+0000 7f4a4b7fe640 1 -- 172.21.15.163:0/126628914 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2335035453 0 0) 0x7f4a4c06f6c0 con 0x7f4a4c06d870 2024-05-08T06:37:30.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.104+0000 7f4a50f03640 1 --2- 172.21.15.163:0/126628914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c15d600 0x7f4a4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.104+0000 7f4a4b7fe640 1 -- 172.21.15.163:0/126628914 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a3c003520 con 0x7f4a4c06d870 2024-05-08T06:37:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.104+0000 7f4a51704640 1 --2- 172.21.15.163:0/126628914 >> v2:172.21.15.192:3300/0 conn(0x7f4a4c06f150 0x7f4a4c15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4b7fe640 1 -- 172.21.15.163:0/126628914 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3455492775 0 0) 0x7f4a3c003520 con 0x7f4a4c06d870 2024-05-08T06:37:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4b7fe640 1 -- 172.21.15.163:0/126628914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c15d600 msgr2=0x7f4a4c06d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4b7fe640 1 --2- 172.21.15.163:0/126628914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c15d600 0x7f4a4c06d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4b7fe640 1 -- 172.21.15.163:0/126628914 >> v2:172.21.15.192:3300/0 conn(0x7f4a4c06f150 msgr2=0x7f4a4c15e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4b7fe640 1 --2- 172.21.15.163:0/126628914 >> v2:172.21.15.192:3300/0 conn(0x7f4a4c06f150 0x7f4a4c15e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4b7fe640 1 -- 172.21.15.163:0/126628914 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c15e100 con 0x7f4a4c06d870 2024-05-08T06:37:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4b7fe640 1 -- 172.21.15.163:0/126628914 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f4a380016f0 con 0x7f4a4c06d870 2024-05-08T06:37:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4b7fe640 1 -- 172.21.15.163:0/126628914 >> v1:172.21.15.163:6789/0 conn(0x7f4a4c06d870 legacy=0x7f4a4c06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4b7fe640 1 --2- 172.21.15.163:0/126628914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a3c003a00 0x7f4a3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4b7fe640 1 --2- 172.21.15.163:0/126628914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c004800 0x7f4a3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4b7fe640 1 --2- 172.21.15.163:0/126628914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c15d600 0x7f4a3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4b7fe640 1 -- 172.21.15.163:0/126628914 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c06dcb0 con 0x7f4a3c003a00 2024-05-08T06:37:30.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4b7fe640 1 -- 172.21.15.163:0/126628914 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c15de20 con 0x7f4a3c004800 2024-05-08T06:37:30.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4b7fe640 1 -- 172.21.15.163:0/126628914 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4a3c00ba40 con 0x7f4a4c15d600 2024-05-08T06:37:30.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a50f03640 1 --2- 172.21.15.163:0/126628914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a3c003a00 0x7f4a3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a4bfff640 1 --2- 172.21.15.163:0/126628914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c004800 0x7f4a3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a51704640 1 --2- 172.21.15.163:0/126628914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c15d600 0x7f4a3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a50f03640 1 -- 172.21.15.163:0/126628914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c15d600 msgr2=0x7f4a3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:30.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a50f03640 1 --2- 172.21.15.163:0/126628914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c15d600 0x7f4a3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a50f03640 1 -- 172.21.15.163:0/126628914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c004800 msgr2=0x7f4a3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:30.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a50f03640 1 --2- 172.21.15.163:0/126628914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c004800 0x7f4a3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.105+0000 7f4a50f03640 1 -- 172.21.15.163:0/126628914 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c003520 con 0x7f4a3c003a00 2024-05-08T06:37:30.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.106+0000 7f4a50f03640 1 --2- 172.21.15.163:0/126628914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a3c003a00 0x7f4a3c003e20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009ac0 tx=0x7f4a3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef21e1f376bc25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.106+0000 7f4a4b7fe640 1 -- 172.21.15.163:0/126628914 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40036070 con 0x7f4a3c003a00 2024-05-08T06:37:30.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.106+0000 7f4a4b7fe640 1 -- 172.21.15.163:0/126628914 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a40031400 con 0x7f4a3c003a00 2024-05-08T06:37:30.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.106+0000 7f4a4b7fe640 1 -- 172.21.15.163:0/126628914 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40009690 con 0x7f4a3c003a00 2024-05-08T06:37:30.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.107+0000 7f4a53167640 1 -- 172.21.15.163:0/126628914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a3c003a00 msgr2=0x7f4a3c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.107+0000 7f4a53167640 1 --2- 172.21.15.163:0/126628914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a3c003a00 0x7f4a3c003e20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009ac0 tx=0x7f4a3c005830 comp rx=0 tx=0).stop 2024-05-08T06:37:30.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.107+0000 7f4a51704640 1 -- 172.21.15.163:0/126628914 reap_dead start 2024-05-08T06:37:30.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.107+0000 7f4a53167640 1 -- 172.21.15.163:0/126628914 shutdown_connections 2024-05-08T06:37:30.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.107+0000 7f4a53167640 1 -- 172.21.15.163:0/126628914 >> 172.21.15.163:0/126628914 conn(0x7f4a4c072ba0 msgr2=0x7f4a4c072f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:30.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.107+0000 7f4a53167640 1 -- 172.21.15.163:0/126628914 shutdown_connections 2024-05-08T06:37:30.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.107+0000 7f4a53167640 1 -- 172.21.15.163:0/126628914 wait complete. 2024-05-08T06:37:30.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.108+0000 7f4a53167640 1 Processor -- start 2024-05-08T06:37:30.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.108+0000 7f4a53167640 1 -- start start 2024-05-08T06:37:30.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a53167640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c003a00 0x7f4a4c1e5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a53167640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c142710 0x7f4a4c0b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a53167640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a4c0b1cd0 0x7f4a4c0b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a53167640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c15d9e0 con 0x7f4a4c0b1cd0 2024-05-08T06:37:30.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a53167640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c06f9a0 con 0x7f4a3c003a00 2024-05-08T06:37:30.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a53167640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4c15db60 con 0x7f4a4c142710 2024-05-08T06:37:30.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a51704640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a4c0b1cd0 0x7f4a4c0b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a51704640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a4c0b1cd0 0x7f4a4c0b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38598/0 (socket says 172.21.15.163:38598) 2024-05-08T06:37:30.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a51704640 1 -- 172.21.15.163:0/3450086751 learned_addr learned my addr 172.21.15.163:0/3450086751 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:30.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a50f03640 1 --2- 172.21.15.163:0/3450086751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c003a00 0x7f4a4c1e5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a4bfff640 1 --2- 172.21.15.163:0/3450086751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c142710 0x7f4a4c0b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a51704640 1 -- 172.21.15.163:0/3450086751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c142710 msgr2=0x7f4a4c0b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a51704640 1 --2- 172.21.15.163:0/3450086751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c142710 0x7f4a4c0b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a51704640 1 -- 172.21.15.163:0/3450086751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c003a00 msgr2=0x7f4a4c1e5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a51704640 1 --2- 172.21.15.163:0/3450086751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c003a00 0x7f4a4c1e5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a51704640 1 -- 172.21.15.163:0/3450086751 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c153150 con 0x7f4a4c0b1cd0 2024-05-08T06:37:30.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a51704640 1 --2- 172.21.15.163:0/3450086751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a4c0b1cd0 0x7f4a4c0b2150 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0724b0 tx=0x7f4a4400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.109+0000 7f4a497fa640 1 -- 172.21.15.163:0/3450086751 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44011e60 con 0x7f4a4c0b1cd0 2024-05-08T06:37:30.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.110+0000 7f4a497fa640 1 -- 172.21.15.163:0/3450086751 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a4401d070 con 0x7f4a4c0b1cd0 2024-05-08T06:37:30.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.110+0000 7f4a53167640 1 -- 172.21.15.163:0/3450086751 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4c153430 con 0x7f4a4c0b1cd0 2024-05-08T06:37:30.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.110+0000 7f4a497fa640 1 -- 172.21.15.163:0/3450086751 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44015650 con 0x7f4a4c0b1cd0 2024-05-08T06:37:30.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.110+0000 7f4a53167640 1 -- 172.21.15.163:0/3450086751 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4c1539f0 con 0x7f4a4c0b1cd0 2024-05-08T06:37:30.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.110+0000 7f4a53167640 1 -- 172.21.15.163:0/3450086751 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a4c06a550 con 0x7f4a4c0b1cd0 2024-05-08T06:37:30.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.111+0000 7f4a497fa640 1 -- 172.21.15.163:0/3450086751 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4a440157f0 con 0x7f4a4c0b1cd0 2024-05-08T06:37:30.113 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.112+0000 7f4a497fa640 1 --2- 172.21.15.163:0/3450086751 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4a300762d0 0x7f4a30078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.112+0000 7f4a50f03640 1 --2- 172.21.15.163:0/3450086751 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4a300762d0 0x7f4a30078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.112+0000 7f4a497fa640 1 -- 172.21.15.163:0/3450086751 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f4a44097d70 con 0x7f4a4c0b1cd0 2024-05-08T06:37:30.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.112+0000 7f4a50f03640 1 --2- 172.21.15.163:0/3450086751 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4a300762d0 0x7f4a30078790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4a40033000 tx=0x7f4a40008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.116+0000 7f4a497fa640 1 -- 172.21.15.163:0/3450086751 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4a44010040 con 0x7f4a4c0b1cd0 2024-05-08T06:37:30.335 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.337+0000 7f4a53167640 1 -- 172.21.15.163:0/3450086751 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a4c153bc0 con 0x7f4a4c0b1cd0 2024-05-08T06:37:30.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.337+0000 7f4a497fa640 1 -- 172.21.15.163:0/3450086751 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5190 (secure 0 0 0) 0x7f4a440616d0 con 0x7f4a4c0b1cd0 2024-05-08T06:37:30.336 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:30.336 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1773142301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1773142301},{"type":"v1","addr":"172.21.15.163:6835","nonce":1773142301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2952930735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2952930735},{"type":"v1","addr":"172.21.15.192:6834","nonce":2952930735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1297249355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1297249355},{"type":"v1","addr":"172.21.15.192:6835","nonce":1297249355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:37:22.365453+0000","modified":"2024-05-08T06:37:24.368498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4732":{"gid":4732,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/880835577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":880835577},{"type":"v1","addr":"172.21.15.163:6837","nonce":880835577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4732,"qdb_cluster":[4732]},"id":2}]} 2024-05-08T06:37:30.337 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 14 2024-05-08T06:37:30.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.339+0000 7f4a53167640 1 -- 172.21.15.163:0/3450086751 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4a300762d0 msgr2=0x7f4a30078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.339+0000 7f4a53167640 1 --2- 172.21.15.163:0/3450086751 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4a300762d0 0x7f4a30078790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4a40033000 tx=0x7f4a40008030 comp rx=0 tx=0).stop 2024-05-08T06:37:30.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.339+0000 7f4a53167640 1 -- 172.21.15.163:0/3450086751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a4c0b1cd0 msgr2=0x7f4a4c0b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.339+0000 7f4a53167640 1 --2- 172.21.15.163:0/3450086751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a4c0b1cd0 0x7f4a4c0b2150 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0724b0 tx=0x7f4a4400f580 comp rx=0 tx=0).stop 2024-05-08T06:37:30.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.339+0000 7f4a53167640 1 -- 172.21.15.163:0/3450086751 shutdown_connections 2024-05-08T06:37:30.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.339+0000 7f4a53167640 1 --2- 172.21.15.163:0/3450086751 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f4a300762d0 0x7f4a30078790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.339+0000 7f4a53167640 1 --2- 172.21.15.163:0/3450086751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a4c0b1cd0 0x7f4a4c0b2150 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.339+0000 7f4a53167640 1 --2- 172.21.15.163:0/3450086751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c142710 0x7f4a4c0b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.339+0000 7f4a53167640 1 --2- 172.21.15.163:0/3450086751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c003a00 0x7f4a4c1e5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.339+0000 7f4a53167640 1 -- 172.21.15.163:0/3450086751 >> 172.21.15.163:0/3450086751 conn(0x7f4a4c072ba0 msgr2=0x7f4a4c0ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:30.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.339+0000 7f4a53167640 1 -- 172.21.15.163:0/3450086751 shutdown_connections 2024-05-08T06:37:30.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.339+0000 7f4a53167640 1 -- 172.21.15.163:0/3450086751 wait complete. 2024-05-08T06:37:30.350 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-08T06:37:30.479 INFO:teuthology.orchestra.run.smithi163.stdout:{"message":"","return_code":0} 2024-05-08T06:37:30.479 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T06:37:30.479 DEBUG:teuthology.orchestra.run.smithi163:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2024-05-08T06:37:30.522 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.523+0000 7f07a3cd1f40 1 Processor -- start 2024-05-08T06:37:30.522 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.523+0000 7f07a3cd1f40 1 -- start start 2024-05-08T06:37:30.527 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.523+0000 7f07a3cd1f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55a0b666c4b0 0x55a0b6669230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.527 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.523+0000 7f07a3cd1f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0b6741a90 0x55a0b6737460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.527 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.523+0000 7f07a3cd1f40 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a0b6741e70 0x55a0b67379a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.527 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.523+0000 7f07a3cd1f40 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a0b666c920 con 0x55a0b6741e70 2024-05-08T06:37:30.527 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.523+0000 7f07a3cd1f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a0b6611f50 con 0x55a0b6741a90 2024-05-08T06:37:30.527 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.523+0000 7f07a3cd1f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55a0b6699b60 con 0x55a0b666c4b0 2024-05-08T06:37:30.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.523+0000 7f07a1fce640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a0b6741e70 0x55a0b67379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.523+0000 7f07a1fce640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a0b6741e70 0x55a0b67379a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38618/0 (socket says 172.21.15.163:38618) 2024-05-08T06:37:30.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.523+0000 7f07a1fce640 1 -- 172.21.15.163:0/855326433 learned_addr learned my addr 172.21.15.163:0/855326433 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:30.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a0fcc640 1 --2- 172.21.15.163:0/855326433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0b6741a90 0x55a0b6737460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a17cd640 1 --2- 172.21.15.163:0/855326433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55a0b666c4b0 0x55a0b6669230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a1fce640 1 -- 172.21.15.163:0/855326433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55a0b666c4b0 msgr2=0x55a0b6669230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:30.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a1fce640 1 --2- 172.21.15.163:0/855326433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55a0b666c4b0 0x55a0b6669230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a1fce640 1 -- 172.21.15.163:0/855326433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0b6741a90 msgr2=0x55a0b6737460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:30.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a1fce640 1 --2- 172.21.15.163:0/855326433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0b6741a90 0x55a0b6737460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a1fce640 1 -- 172.21.15.163:0/855326433 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0b67381c0 con 0x55a0b6741e70 2024-05-08T06:37:30.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a0fcc640 1 --2- 172.21.15.163:0/855326433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0b6741a90 0x55a0b6737460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:30.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a1fce640 1 --2- 172.21.15.163:0/855326433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a0b6741e70 0x55a0b67379a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f079c001690 tx=0x7f079c001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7efe6b48e6a6b40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07937fe640 1 -- 172.21.15.163:0/855326433 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c017070 con 0x55a0b6741e70 2024-05-08T06:37:30.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07937fe640 1 -- 172.21.15.163:0/855326433 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079c00f710 con 0x55a0b6741e70 2024-05-08T06:37:30.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07937fe640 1 -- 172.21.15.163:0/855326433 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c00fb00 con 0x55a0b6741e70 2024-05-08T06:37:30.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/855326433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a0b6741e70 msgr2=0x55a0b67379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a3cd1f40 1 --2- 172.21.15.163:0/855326433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a0b6741e70 0x55a0b67379a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f079c001690 tx=0x7f079c001b50 comp rx=0 tx=0).stop 2024-05-08T06:37:30.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/855326433 shutdown_connections 2024-05-08T06:37:30.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a3cd1f40 1 --2- 172.21.15.163:0/855326433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a0b6741e70 0x55a0b67379a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a3cd1f40 1 --2- 172.21.15.163:0/855326433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0b6741a90 0x55a0b6737460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a3cd1f40 1 --2- 172.21.15.163:0/855326433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55a0b666c4b0 0x55a0b6669230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/855326433 >> 172.21.15.163:0/855326433 conn(0x55a0b66b4f10 msgr2=0x55a0b66b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:30.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/855326433 shutdown_connections 2024-05-08T06:37:30.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.524+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/855326433 wait complete. 2024-05-08T06:37:30.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.525+0000 7f07a3cd1f40 1 Processor -- start 2024-05-08T06:37:30.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.525+0000 7f07a3cd1f40 1 -- start start 2024-05-08T06:37:30.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a3cd1f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0b666c4b0 0x55a0b6740350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a3cd1f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55a0b6741a90 0x55a0b66aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a3cd1f40 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a0b6741e70 0x55a0b67c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a3cd1f40 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a0b66abcc0 con 0x55a0b6741e70 2024-05-08T06:37:30.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a3cd1f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a0b66aba70 con 0x55a0b666c4b0 2024-05-08T06:37:30.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a3cd1f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55a0b66abfa0 con 0x55a0b6741a90 2024-05-08T06:37:30.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a1fce640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a0b6741e70 0x55a0b67c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a1fce640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a0b6741e70 0x55a0b67c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38620/0 (socket says 172.21.15.163:38620) 2024-05-08T06:37:30.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a1fce640 1 -- 172.21.15.163:0/3601834656 learned_addr learned my addr 172.21.15.163:0/3601834656 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:30.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a17cd640 1 --2- 172.21.15.163:0/3601834656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0b666c4b0 0x55a0b6740350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a0fcc640 1 --2- 172.21.15.163:0/3601834656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55a0b6741a90 0x55a0b66aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a1fce640 1 -- 172.21.15.163:0/3601834656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55a0b6741a90 msgr2=0x55a0b66aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a1fce640 1 --2- 172.21.15.163:0/3601834656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55a0b6741a90 0x55a0b66aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a1fce640 1 -- 172.21.15.163:0/3601834656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0b666c4b0 msgr2=0x55a0b6740350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a1fce640 1 --2- 172.21.15.163:0/3601834656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0b666c4b0 0x55a0b6740350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a1fce640 1 -- 172.21.15.163:0/3601834656 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0b66ab1b0 con 0x55a0b6741e70 2024-05-08T06:37:30.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07a1fce640 1 --2- 172.21.15.163:0/3601834656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a0b6741e70 0x55a0b67c6fa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55a0b6739910 tx=0x7f079c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07927fc640 1 -- 172.21.15.163:0/3601834656 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c017030 con 0x55a0b6741e70 2024-05-08T06:37:30.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.526+0000 7f07927fc640 1 -- 172.21.15.163:0/3601834656 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079c006030 con 0x55a0b6741e70 2024-05-08T06:37:30.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.527+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/3601834656 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a0b6696340 con 0x55a0b6741e70 2024-05-08T06:37:30.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.527+0000 7f07927fc640 1 -- 172.21.15.163:0/3601834656 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c011650 con 0x55a0b6741e70 2024-05-08T06:37:30.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.527+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/3601834656 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0b66ae540 con 0x55a0b6741e70 2024-05-08T06:37:30.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.528+0000 7f07927fc640 1 -- 172.21.15.163:0/3601834656 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f079c0117f0 con 0x55a0b6741e70 2024-05-08T06:37:30.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.528+0000 7f07927fc640 1 --2- 172.21.15.163:0/3601834656 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0784074cb0 0x7f0784077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.529+0000 7f07a17cd640 1 --2- 172.21.15.163:0/3601834656 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0784074cb0 0x7f0784077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.529+0000 7f07927fc640 1 -- 172.21.15.163:0/3601834656 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f079c0971c0 con 0x55a0b6741e70 2024-05-08T06:37:30.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.529+0000 7f07a3cd1f40 1 --2- 172.21.15.163:0/3601834656 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x55a0b66af280 0x55a0b67cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.529+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/3601834656 --> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a0b6735a60 con 0x55a0b66af280 2024-05-08T06:37:30.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.529+0000 7f07a0fcc640 1 --2- 172.21.15.163:0/3601834656 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x55a0b66af280 0x55a0b67cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.529+0000 7f07a17cd640 1 --2- 172.21.15.163:0/3601834656 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0784074cb0 0x7f0784077170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0794058000 tx=0x7f079400a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.529+0000 7f07a0fcc640 1 --2- 172.21.15.163:0/3601834656 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x55a0b66af280 0x55a0b67cb730 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.530+0000 7f07a0fcc640 1 -- 172.21.15.163:0/3601834656 <== osd.3 v2:172.21.15.163:6816/127592521 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys out=19b] v0'0 uv10 ondisk = 0) ==== 164+0+19 (crc 0 0 0) 0x7f0798009a90 con 0x55a0b66af280 2024-05-08T06:37:30.534 INFO:teuthology.orchestra.run.smithi163.stdout:alpha_head 2024-05-08T06:37:30.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.530+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/3601834656 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x55a0b66af280 msgr2=0x55a0b67cb730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.530+0000 7f07a3cd1f40 1 --2- 172.21.15.163:0/3601834656 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x55a0b66af280 0x55a0b67cb730 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.530+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/3601834656 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0784074cb0 msgr2=0x7f0784077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.530+0000 7f07a3cd1f40 1 --2- 172.21.15.163:0/3601834656 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f0784074cb0 0x7f0784077170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0794058000 tx=0x7f079400a000 comp rx=0 tx=0).stop 2024-05-08T06:37:30.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.530+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/3601834656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a0b6741e70 msgr2=0x55a0b67c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.530+0000 7f07a3cd1f40 1 --2- 172.21.15.163:0/3601834656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a0b6741e70 0x55a0b67c6fa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55a0b6739910 tx=0x7f079c001250 comp rx=0 tx=0).stop 2024-05-08T06:37:30.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.530+0000 7f07a1fce640 1 -- 172.21.15.163:0/3601834656 reap_dead start 2024-05-08T06:37:30.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.530+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/3601834656 shutdown_connections 2024-05-08T06:37:30.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.530+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/3601834656 >> 172.21.15.163:0/3601834656 conn(0x55a0b66b4f10 msgr2=0x55a0b666b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:30.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.530+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/3601834656 shutdown_connections 2024-05-08T06:37:30.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.530+0000 7f07a3cd1f40 1 -- 172.21.15.163:0/3601834656 wait complete. 2024-05-08T06:37:30.536 DEBUG:teuthology.orchestra.run.smithi163:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2024-05-08T06:37:30.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.620+0000 7fa0f51c9f40 1 Processor -- start 2024-05-08T06:37:30.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.620+0000 7fa0f51c9f40 1 -- start start 2024-05-08T06:37:30.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.620+0000 7fa0f51c9f40 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fd2a128f0 0x561fd2adf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.620+0000 7fa0f51c9f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561fd2ae8210 0x561fd2ae59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.620+0000 7fa0f51c9f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561fd2ae8b30 0x561fd2ae5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.620+0000 7fa0f51c9f40 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561fd2a14910 con 0x561fd2a128f0 2024-05-08T06:37:30.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.620+0000 7fa0f51c9f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561fd29b9f50 con 0x561fd2ae8210 2024-05-08T06:37:30.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.620+0000 7fa0f51c9f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x561fd2a41f20 con 0x561fd2ae8b30 2024-05-08T06:37:30.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.620+0000 7fa0f2cc3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fd2a128f0 0x561fd2adf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f2cc3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fd2a128f0 0x561fd2adf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38640/0 (socket says 172.21.15.163:38640) 2024-05-08T06:37:30.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f2cc3640 1 -- 172.21.15.163:0/4140322400 learned_addr learned my addr 172.21.15.163:0/4140322400 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:30.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f2cc3640 1 -- 172.21.15.163:0/4140322400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561fd2ae8b30 msgr2=0x561fd2ae5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:30.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f34c4640 1 --2- 172.21.15.163:0/4140322400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561fd2ae8b30 0x561fd2ae5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f2cc3640 1 --2- 172.21.15.163:0/4140322400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561fd2ae8b30 0x561fd2ae5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f24c2640 1 --2- 172.21.15.163:0/4140322400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561fd2ae8210 0x561fd2ae59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f2cc3640 1 -- 172.21.15.163:0/4140322400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561fd2ae8210 msgr2=0x561fd2ae59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:30.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f2cc3640 1 --2- 172.21.15.163:0/4140322400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561fd2ae8210 0x561fd2ae59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f2cc3640 1 -- 172.21.15.163:0/4140322400 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd2a3e5d0 con 0x561fd2a128f0 2024-05-08T06:37:30.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f34c4640 1 --2- 172.21.15.163:0/4140322400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561fd2ae8b30 0x561fd2ae5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:30.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f24c2640 1 --2- 172.21.15.163:0/4140322400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561fd2ae8210 0x561fd2ae59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:30.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f2cc3640 1 --2- 172.21.15.163:0/4140322400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fd2a128f0 0x561fd2adf480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4000c00 tx=0x7fa0e4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421ed7fbe5716b9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f14c0640 1 -- 172.21.15.163:0/4140322400 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4063070 con 0x561fd2a128f0 2024-05-08T06:37:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f14c0640 1 -- 172.21.15.163:0/4140322400 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e4055890 con 0x561fd2a128f0 2024-05-08T06:37:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.621+0000 7fa0f14c0640 1 -- 172.21.15.163:0/4140322400 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4055c80 con 0x561fd2a128f0 2024-05-08T06:37:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.622+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4140322400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fd2a128f0 msgr2=0x561fd2adf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.622+0000 7fa0f51c9f40 1 --2- 172.21.15.163:0/4140322400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fd2a128f0 0x561fd2adf480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4000c00 tx=0x7fa0e4051000 comp rx=0 tx=0).stop 2024-05-08T06:37:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.622+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4140322400 shutdown_connections 2024-05-08T06:37:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.622+0000 7fa0f51c9f40 1 --2- 172.21.15.163:0/4140322400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561fd2ae8b30 0x561fd2ae5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.622+0000 7fa0f51c9f40 1 --2- 172.21.15.163:0/4140322400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561fd2ae8210 0x561fd2ae59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.622+0000 7fa0f51c9f40 1 --2- 172.21.15.163:0/4140322400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fd2a128f0 0x561fd2adf480 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.622+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4140322400 >> 172.21.15.163:0/4140322400 conn(0x561fd2a5cf10 msgr2=0x561fd2a5d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.622+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4140322400 shutdown_connections 2024-05-08T06:37:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.622+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4140322400 wait complete. 2024-05-08T06:37:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.622+0000 7fa0f51c9f40 1 Processor -- start 2024-05-08T06:37:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.622+0000 7fa0f51c9f40 1 -- start start 2024-05-08T06:37:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f51c9f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561fd2a128f0 0x561fd2ae3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f51c9f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561fd2ae8210 0x561fd2adc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f51c9f40 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fd2ae8b30 0x561fd2b7de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f51c9f40 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561fd2ae6e60 con 0x561fd2ae8b30 2024-05-08T06:37:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f51c9f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561fd2ae6b80 con 0x561fd2a128f0 2024-05-08T06:37:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f51c9f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x561fd2adfd70 con 0x561fd2ae8210 2024-05-08T06:37:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f34c4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fd2ae8b30 0x561fd2b7de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f34c4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fd2ae8b30 0x561fd2b7de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38644/0 (socket says 172.21.15.163:38644) 2024-05-08T06:37:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f34c4640 1 -- 172.21.15.163:0/4195384367 learned_addr learned my addr 172.21.15.163:0/4195384367 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f2cc3640 1 --2- 172.21.15.163:0/4195384367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561fd2a128f0 0x561fd2ae3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f24c2640 1 --2- 172.21.15.163:0/4195384367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561fd2ae8210 0x561fd2adc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f34c4640 1 -- 172.21.15.163:0/4195384367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561fd2ae8210 msgr2=0x561fd2adc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f34c4640 1 --2- 172.21.15.163:0/4195384367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561fd2ae8210 0x561fd2adc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f34c4640 1 -- 172.21.15.163:0/4195384367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561fd2a128f0 msgr2=0x561fd2ae3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f34c4640 1 --2- 172.21.15.163:0/4195384367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561fd2a128f0 0x561fd2ae3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.623+0000 7fa0f34c4640 1 -- 172.21.15.163:0/4195384367 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd2ae2eb0 con 0x561fd2ae8b30 2024-05-08T06:37:30.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.624+0000 7fa0f34c4640 1 --2- 172.21.15.163:0/4195384367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fd2ae8b30 0x561fd2b7de80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00cfc0 tx=0x7fa0ec00e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.624+0000 7fa0dbfff640 1 -- 172.21.15.163:0/4195384367 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec014410 con 0x561fd2ae8b30 2024-05-08T06:37:30.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.624+0000 7fa0dbfff640 1 -- 172.21.15.163:0/4195384367 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ec0145d0 con 0x561fd2ae8b30 2024-05-08T06:37:30.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.624+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4195384367 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561fd2ae3150 con 0x561fd2ae8b30 2024-05-08T06:37:30.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.624+0000 7fa0dbfff640 1 -- 172.21.15.163:0/4195384367 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec0149c0 con 0x561fd2ae8b30 2024-05-08T06:37:30.627 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.624+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4195384367 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd2ae3710 con 0x561fd2ae8b30 2024-05-08T06:37:30.628 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.625+0000 7fa0dbfff640 1 -- 172.21.15.163:0/4195384367 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa0ec01b030 con 0x561fd2ae8b30 2024-05-08T06:37:30.628 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.626+0000 7fa0dbfff640 1 --2- 172.21.15.163:0/4195384367 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa0e0074ce0 0x7fa0e00771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.628 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.626+0000 7fa0f2cc3640 1 --2- 172.21.15.163:0/4195384367 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa0e0074ce0 0x7fa0e00771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.626+0000 7fa0dbfff640 1 -- 172.21.15.163:0/4195384367 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fa0ec095e70 con 0x561fd2ae8b30 2024-05-08T06:37:30.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.626+0000 7fa0f51c9f40 1 --2- 172.21.15.163:0/4195384367 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x561fd2b7e8d0 0x561fd2a40b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.626+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4195384367 --> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] -- osd_op(unknown.0.0:1 4.1f 4:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561fd2a41090 con 0x561fd2b7e8d0 2024-05-08T06:37:30.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.626+0000 7fa0f2cc3640 1 --2- 172.21.15.163:0/4195384367 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa0e0074ce0 0x7fa0e00771a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4067000 tx=0x7fa0e406a040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.626+0000 7fa0f24c2640 1 --2- 172.21.15.163:0/4195384367 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x561fd2b7e8d0 0x561fd2a40b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.627+0000 7fa0f24c2640 1 --2- 172.21.15.163:0/4195384367 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x561fd2b7e8d0 0x561fd2a40b50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.629 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.627+0000 7fa0f24c2640 1 -- 172.21.15.163:0/4195384367 <== osd.6 v2:172.21.15.192:6816/1074108675 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=19b] v0'0 uv6 ondisk = 0) ==== 154+0+19 (crc 0 0 0) 0x7fa0e800f6c0 con 0x561fd2b7e8d0 2024-05-08T06:37:30.629 INFO:teuthology.orchestra.run.smithi163.stdout:mydir_head 2024-05-08T06:37:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.627+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4195384367 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x561fd2b7e8d0 msgr2=0x561fd2a40b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.627+0000 7fa0f51c9f40 1 --2- 172.21.15.163:0/4195384367 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x561fd2b7e8d0 0x561fd2a40b50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.628+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4195384367 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa0e0074ce0 msgr2=0x7fa0e00771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.628+0000 7fa0f51c9f40 1 --2- 172.21.15.163:0/4195384367 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa0e0074ce0 0x7fa0e00771a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4067000 tx=0x7fa0e406a040 comp rx=0 tx=0).stop 2024-05-08T06:37:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.628+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4195384367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fd2ae8b30 msgr2=0x561fd2b7de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.628+0000 7fa0f51c9f40 1 --2- 172.21.15.163:0/4195384367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561fd2ae8b30 0x561fd2b7de80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00cfc0 tx=0x7fa0ec00e670 comp rx=0 tx=0).stop 2024-05-08T06:37:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.628+0000 7fa0f34c4640 1 -- 172.21.15.163:0/4195384367 reap_dead start 2024-05-08T06:37:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.628+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4195384367 shutdown_connections 2024-05-08T06:37:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.628+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4195384367 >> 172.21.15.163:0/4195384367 conn(0x561fd2a5cf10 msgr2=0x561fd2a5d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.628+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4195384367 shutdown_connections 2024-05-08T06:37:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.628+0000 7fa0f51c9f40 1 -- 172.21.15.163:0/4195384367 wait complete. 2024-05-08T06:37:30.632 DEBUG:teuthology.orchestra.run.smithi163:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-08T06:37:30.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.715+0000 7fe932623f40 1 Processor -- start 2024-05-08T06:37:30.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.715+0000 7fe932623f40 1 -- start start 2024-05-08T06:37:30.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe932623f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x559e0cb85e50 0x559e0cc5b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe932623f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e0cb871e0 0x559e0cc5b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe932623f40 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e0cc5d210 0x559e0cc559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe932623f40 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559e0cb87b90 con 0x559e0cc5d210 2024-05-08T06:37:30.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe932623f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559e0cb2ef50 con 0x559e0cb871e0 2024-05-08T06:37:30.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe932623f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x559e0cb89910 con 0x559e0cb85e50 2024-05-08T06:37:30.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe93091e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e0cc5d210 0x559e0cc559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe93091e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e0cc5d210 0x559e0cc559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38658/0 (socket says 172.21.15.163:38658) 2024-05-08T06:37:30.719 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe93091e640 1 -- 172.21.15.163:0/1228124759 learned_addr learned my addr 172.21.15.163:0/1228124759 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:30.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe92f91c640 1 --2- 172.21.15.163:0/1228124759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e0cb871e0 0x559e0cc5b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe93091e640 1 -- 172.21.15.163:0/1228124759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x559e0cb85e50 msgr2=0x559e0cc5b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:30.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe93011d640 1 --2- 172.21.15.163:0/1228124759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x559e0cb85e50 0x559e0cc5b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe93091e640 1 --2- 172.21.15.163:0/1228124759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x559e0cb85e50 0x559e0cc5b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe93091e640 1 -- 172.21.15.163:0/1228124759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e0cb871e0 msgr2=0x559e0cc5b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:30.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe93091e640 1 --2- 172.21.15.163:0/1228124759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e0cb871e0 0x559e0cc5b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe93091e640 1 -- 172.21.15.163:0/1228124759 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0cbb1cf0 con 0x559e0cc5d210 2024-05-08T06:37:30.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe92f91c640 1 --2- 172.21.15.163:0/1228124759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e0cb871e0 0x559e0cc5b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:30.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.716+0000 7fe93011d640 1 --2- 172.21.15.163:0/1228124759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x559e0cb85e50 0x559e0cc5b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:30.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.717+0000 7fe93091e640 1 --2- 172.21.15.163:0/1228124759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e0cc5d210 0x559e0cc559c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe92800b5e0 tx=0x7fe92800baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c501965cc27ff2f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.717+0000 7fe92e91a640 1 -- 172.21.15.163:0/1228124759 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928014020 con 0x559e0cc5d210 2024-05-08T06:37:30.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.717+0000 7fe92e91a640 1 -- 172.21.15.163:0/1228124759 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe928010820 con 0x559e0cc5d210 2024-05-08T06:37:30.720 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.717+0000 7fe92e91a640 1 -- 172.21.15.163:0/1228124759 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928010c10 con 0x559e0cc5d210 2024-05-08T06:37:30.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.717+0000 7fe932623f40 1 -- 172.21.15.163:0/1228124759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e0cc5d210 msgr2=0x559e0cc559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.717+0000 7fe932623f40 1 --2- 172.21.15.163:0/1228124759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e0cc5d210 0x559e0cc559c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe92800b5e0 tx=0x7fe92800baa0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.717+0000 7fe932623f40 1 -- 172.21.15.163:0/1228124759 shutdown_connections 2024-05-08T06:37:30.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.717+0000 7fe932623f40 1 --2- 172.21.15.163:0/1228124759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e0cc5d210 0x559e0cc559c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.717+0000 7fe932623f40 1 --2- 172.21.15.163:0/1228124759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e0cb871e0 0x559e0cc5b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.717+0000 7fe932623f40 1 --2- 172.21.15.163:0/1228124759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x559e0cb85e50 0x559e0cc5b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.717+0000 7fe932623f40 1 -- 172.21.15.163:0/1228124759 >> 172.21.15.163:0/1228124759 conn(0x559e0cbd1f10 msgr2=0x559e0cc552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:30.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.717+0000 7fe932623f40 1 -- 172.21.15.163:0/1228124759 shutdown_connections 2024-05-08T06:37:30.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.717+0000 7fe932623f40 1 -- 172.21.15.163:0/1228124759 wait complete. 2024-05-08T06:37:30.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.718+0000 7fe932623f40 1 Processor -- start 2024-05-08T06:37:30.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.718+0000 7fe932623f40 1 -- start start 2024-05-08T06:37:30.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe932623f40 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e0cb85e50 0x559e0cbb52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.721 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe932623f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e0cb871e0 0x559e0ccf89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe932623f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x559e0cc5d210 0x559e0ccf8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe932623f40 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559e0cc5c5e0 con 0x559e0cb85e50 2024-05-08T06:37:30.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe932623f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559e0cbd2830 con 0x559e0cb871e0 2024-05-08T06:37:30.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe932623f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x559e0cc5f600 con 0x559e0cc5d210 2024-05-08T06:37:30.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe93011d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e0cb85e50 0x559e0cbb52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe93011d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e0cb85e50 0x559e0cbb52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38672/0 (socket says 172.21.15.163:38672) 2024-05-08T06:37:30.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe93011d640 1 -- 172.21.15.163:0/948899845 learned_addr learned my addr 172.21.15.163:0/948899845 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:30.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe93011d640 1 -- 172.21.15.163:0/948899845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x559e0cc5d210 msgr2=0x559e0ccf8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe93011d640 1 --2- 172.21.15.163:0/948899845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x559e0cc5d210 0x559e0ccf8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe93011d640 1 -- 172.21.15.163:0/948899845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e0cb871e0 msgr2=0x559e0ccf89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe93011d640 1 --2- 172.21.15.163:0/948899845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e0cb871e0 0x559e0ccf89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe93011d640 1 -- 172.21.15.163:0/948899845 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0cc544e0 con 0x559e0cb85e50 2024-05-08T06:37:30.722 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe93011d640 1 --2- 172.21.15.163:0/948899845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e0cb85e50 0x559e0cbb52d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe92000cf40 tx=0x7fe92000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.723 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe92d918640 1 -- 172.21.15.163:0/948899845 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920014070 con 0x559e0cb85e50 2024-05-08T06:37:30.723 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe92d918640 1 -- 172.21.15.163:0/948899845 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92000ae30 con 0x559e0cb85e50 2024-05-08T06:37:30.723 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe932623f40 1 -- 172.21.15.163:0/948899845 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559e0cc5fd00 con 0x559e0cb85e50 2024-05-08T06:37:30.723 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe92d918640 1 -- 172.21.15.163:0/948899845 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920010880 con 0x559e0cb85e50 2024-05-08T06:37:30.723 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.719+0000 7fe932623f40 1 -- 172.21.15.163:0/948899845 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e0cc519d0 con 0x559e0cb85e50 2024-05-08T06:37:30.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.721+0000 7fe92d918640 1 -- 172.21.15.163:0/948899845 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe92000e040 con 0x559e0cb85e50 2024-05-08T06:37:30.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.721+0000 7fe92d918640 1 --2- 172.21.15.163:0/948899845 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe91c074ce0 0x7fe91c0771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.721+0000 7fe92d918640 1 -- 172.21.15.163:0/948899845 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fe920095f10 con 0x559e0cb85e50 2024-05-08T06:37:30.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.721+0000 7fe92f91c640 1 --2- 172.21.15.163:0/948899845 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe91c074ce0 0x7fe91c0771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.721+0000 7fe932623f40 1 --2- 172.21.15.163:0/948899845 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x559e0cbb5810 0x559e0cbcbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.721+0000 7fe932623f40 1 -- 172.21.15.163:0/948899845 --> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559e0cbb7db0 con 0x559e0cbb5810 2024-05-08T06:37:30.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.722+0000 7fe93091e640 1 --2- 172.21.15.163:0/948899845 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x559e0cbb5810 0x559e0cbcbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.722+0000 7fe92f91c640 1 --2- 172.21.15.163:0/948899845 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe91c074ce0 0x7fe91c0771a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe924057000 tx=0x7fe924051000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.722+0000 7fe93091e640 1 --2- 172.21.15.163:0/948899845 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x559e0cbb5810 0x559e0cbcbac0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.722+0000 7fe93091e640 1 -- 172.21.15.163:0/948899845 <== osd.5 v2:172.21.15.192:6800/2911825781 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (92) out=92b] v0'0 uv6 ondisk = 0) ==== 164+0+92 (crc 0 0 0) 0x7fe928010a50 con 0x559e0cbb5810 2024-05-08T06:37:30.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.722+0000 7fe932623f40 1 -- 172.21.15.163:0/948899845 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x559e0cbb5810 msgr2=0x559e0cbcbac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.722+0000 7fe932623f40 1 --2- 172.21.15.163:0/948899845 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x559e0cbb5810 0x559e0cbcbac0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.722+0000 7fe932623f40 1 -- 172.21.15.163:0/948899845 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe91c074ce0 msgr2=0x7fe91c0771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.722+0000 7fe932623f40 1 --2- 172.21.15.163:0/948899845 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe91c074ce0 0x7fe91c0771a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe924057000 tx=0x7fe924051000 comp rx=0 tx=0).stop 2024-05-08T06:37:30.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.722+0000 7fe932623f40 1 -- 172.21.15.163:0/948899845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e0cb85e50 msgr2=0x559e0cbb52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.722+0000 7fe932623f40 1 --2- 172.21.15.163:0/948899845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559e0cb85e50 0x559e0cbb52d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe92000cf40 tx=0x7fe92000a660 comp rx=0 tx=0).stop 2024-05-08T06:37:30.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.722+0000 7fe93091e640 1 -- 172.21.15.163:0/948899845 reap_dead start 2024-05-08T06:37:30.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.723+0000 7fe932623f40 1 -- 172.21.15.163:0/948899845 shutdown_connections 2024-05-08T06:37:30.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.723+0000 7fe932623f40 1 -- 172.21.15.163:0/948899845 >> 172.21.15.163:0/948899845 conn(0x559e0cbd1f10 msgr2=0x559e0cc5aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:30.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.723+0000 7fe932623f40 1 -- 172.21.15.163:0/948899845 shutdown_connections 2024-05-08T06:37:30.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.723+0000 7fe932623f40 1 -- 172.21.15.163:0/948899845 wait complete. 2024-05-08T06:37:30.727 INFO:teuthology.orchestra.run.smithi163.stdout:Valphamydir 2024-05-08T06:37:30.727 DEBUG:teuthology.orchestra.run.smithi163:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-08T06:37:30.866 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:37:30.866 INFO:teuthology.orchestra.run.smithi163.stdout: "ino": 1099511627777, 2024-05-08T06:37:30.866 INFO:teuthology.orchestra.run.smithi163.stdout: "ancestors": [ 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout: "dirino": 1099511627776, 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout: "dname": "alpha", 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout: "version": 6 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout: { 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout: "dirino": 1, 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout: "dname": "mydir", 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout: "version": 8 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout: ], 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout: "pool": 5, 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout: "old_pools": [] 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:37:30.867 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:30.877 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T06:37:30.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.979+0000 7fc631b6a640 1 Processor -- start 2024-05-08T06:37:30.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.979+0000 7fc631b6a640 1 -- start start 2024-05-08T06:37:30.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.980+0000 7fc631b6a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc62c15f500 0x7fc62c06f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.980+0000 7fc631b6a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc62c0b7140 0x7fc62c073b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.980+0000 7fc631b6a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc62c0b0fc0 con 0x7fc62c0b3f60 2024-05-08T06:37:30.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.980+0000 7fc631b6a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fc62c0b18f0 con 0x7fc62c0b7140 2024-05-08T06:37:30.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.980+0000 7fc631b6a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc62c06fa10 con 0x7fc62c15f500 2024-05-08T06:37:30.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.980+0000 7fc62affd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc62c0b7140 0x7fc62c073b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.980+0000 7fc62affd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc62c0b7140 0x7fc62c073b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38678/0 (socket says 172.21.15.163:38678) 2024-05-08T06:37:30.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.980+0000 7fc62affd640 1 -- 172.21.15.163:0/2802856824 learned_addr learned my addr 172.21.15.163:0/2802856824 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:30.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.981+0000 7fc62b7fe640 1 --2- 172.21.15.163:0/2802856824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc62c15f500 0x7fc62c06f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.981+0000 7fc62affd640 1 -- 172.21.15.163:0/2802856824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc62c15f500 msgr2=0x7fc62c06f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:30.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.981+0000 7fc62affd640 1 --2- 172.21.15.163:0/2802856824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc62c15f500 0x7fc62c06f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.981+0000 7fc62affd640 1 -- 172.21.15.163:0/2802856824 >> v1:172.21.15.192:6789/0 conn(0x7fc62c0b3f60 legacy=0x7fc62c15fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.981 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.981+0000 7fc62affd640 1 -- 172.21.15.163:0/2802856824 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62c0b12a0 con 0x7fc62c0b7140 2024-05-08T06:37:30.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.981+0000 7fc62affd640 1 --2- 172.21.15.163:0/2802856824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc62c0b7140 0x7fc62c073b10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc620009e50 tx=0x7fc62002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa5a10eb54d0163 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.982+0000 7fc62a7fc640 1 -- 172.21.15.163:0/2802856824 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620036070 con 0x7fc62c0b7140 2024-05-08T06:37:30.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.982+0000 7fc62a7fc640 1 -- 172.21.15.163:0/2802856824 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc620031600 con 0x7fc62c0b7140 2024-05-08T06:37:30.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.982+0000 7fc62a7fc640 1 -- 172.21.15.163:0/2802856824 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6200319d0 con 0x7fc62c0b7140 2024-05-08T06:37:30.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.982+0000 7fc631b6a640 1 -- 172.21.15.163:0/2802856824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc62c0b7140 msgr2=0x7fc62c073b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.982+0000 7fc631b6a640 1 --2- 172.21.15.163:0/2802856824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc62c0b7140 0x7fc62c073b10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc620009e50 tx=0x7fc62002a710 comp rx=0 tx=0).stop 2024-05-08T06:37:30.982 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.982+0000 7fc631b6a640 1 -- 172.21.15.163:0/2802856824 shutdown_connections 2024-05-08T06:37:30.983 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.982+0000 7fc631b6a640 1 --2- 172.21.15.163:0/2802856824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc62c0b7140 0x7fc62c073b10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.983 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.982+0000 7fc631b6a640 1 --2- 172.21.15.163:0/2802856824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc62c15f500 0x7fc62c06f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.983 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.982+0000 7fc631b6a640 1 -- 172.21.15.163:0/2802856824 >> 172.21.15.163:0/2802856824 conn(0x7fc62c06d610 msgr2=0x7fc62c06d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:30.983 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.982+0000 7fc631b6a640 1 -- 172.21.15.163:0/2802856824 shutdown_connections 2024-05-08T06:37:30.983 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.982+0000 7fc631b6a640 1 -- 172.21.15.163:0/2802856824 wait complete. 2024-05-08T06:37:30.983 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.983+0000 7fc631b6a640 1 Processor -- start 2024-05-08T06:37:30.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.984+0000 7fc631b6a640 1 -- start start 2024-05-08T06:37:30.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.984+0000 7fc631b6a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc62c0b3f60 0x7fc62c14cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.984+0000 7fc631b6a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc62c0b7140 0x7fc62c151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.984+0000 7fc631b6a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc62c15f500 0x7fc62c15d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.984+0000 7fc631b6a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c0b43d0 con 0x7fc62c0b3f60 2024-05-08T06:37:30.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.984+0000 7fc631b6a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c074050 con 0x7fc62c0b7140 2024-05-08T06:37:30.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.984+0000 7fc631b6a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc62c0b4550 con 0x7fc62c15f500 2024-05-08T06:37:30.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.984+0000 7fc62b7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc62c0b3f60 0x7fc62c14cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.984+0000 7fc62b7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc62c0b3f60 0x7fc62c14cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38690/0 (socket says 172.21.15.163:38690) 2024-05-08T06:37:30.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.984+0000 7fc62b7fe640 1 -- 172.21.15.163:0/496042332 learned_addr learned my addr 172.21.15.163:0/496042332 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:30.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc62affd640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc62c0b7140 0x7fc62c151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc62bfff640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc62c15f500 0x7fc62c15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc62b7fe640 1 -- 172.21.15.163:0/496042332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc62c15f500 msgr2=0x7fc62c15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc62b7fe640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc62c15f500 0x7fc62c15d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc62b7fe640 1 -- 172.21.15.163:0/496042332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc62c0b7140 msgr2=0x7fc62c151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:30.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc62b7fe640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc62c0b7140 0x7fc62c151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:30.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc62b7fe640 1 -- 172.21.15.163:0/496042332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62c1f5c50 con 0x7fc62c0b3f60 2024-05-08T06:37:30.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc62affd640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc62c0b7140 0x7fc62c151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:30.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc62bfff640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc62c15f500 0x7fc62c15d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:30.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc62b7fe640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc62c0b3f60 0x7fc62c14cac0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00e8a0 tx=0x7fc61c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc613fff640 1 -- 172.21.15.163:0/496042332 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c017070 con 0x7fc62c0b3f60 2024-05-08T06:37:30.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc613fff640 1 -- 172.21.15.163:0/496042332 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc61c00c940 con 0x7fc62c0b3f60 2024-05-08T06:37:30.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc613fff640 1 -- 172.21.15.163:0/496042332 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c00cd30 con 0x7fc62c0b3f60 2024-05-08T06:37:30.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc631b6a640 1 -- 172.21.15.163:0/496042332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62c1f4cf0 con 0x7fc62c0b3f60 2024-05-08T06:37:30.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.985+0000 7fc631b6a640 1 -- 172.21.15.163:0/496042332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62c1f52b0 con 0x7fc62c0b3f60 2024-05-08T06:37:30.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.986+0000 7fc631b6a640 1 -- 172.21.15.163:0/496042332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc62c06a4b0 con 0x7fc62c0b3f60 2024-05-08T06:37:30.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.987+0000 7fc613fff640 1 -- 172.21.15.163:0/496042332 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc61c01a070 con 0x7fc62c0b3f60 2024-05-08T06:37:30.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.987+0000 7fc613fff640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc6180762d0 0x7fc618078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:30.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.987+0000 7fc613fff640 1 -- 172.21.15.163:0/496042332 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fc61c097450 con 0x7fc62c0b3f60 2024-05-08T06:37:30.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.987+0000 7fc62affd640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc6180762d0 0x7fc618078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:30.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.988+0000 7fc62affd640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc6180762d0 0x7fc618078790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc620033000 tx=0x7fc620009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:30.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:30.992+0000 7fc613fff640 1 -- 172.21.15.163:0/496042332 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc61c060e40 con 0x7fc62c0b3f60 2024-05-08T06:37:31.213 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.214+0000 7fc631b6a640 1 -- 172.21.15.163:0/496042332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc62c15ee80 con 0x7fc62c0b3f60 2024-05-08T06:37:31.214 INFO:tasks.ceph.mon.a.smithi163.stderr:2024-05-08T06:37:31.215+0000 7fa54ba64640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:37:31.215 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.216+0000 7fc613fff640 1 -- 172.21.15.163:0/496042332 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) ==== 161+0+0 (secure 0 0 0) 0x7fc61c060740 con 0x7fc62c0b3f60 2024-05-08T06:37:31.215 INFO:teuthology.orchestra.run.smithi163.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T06:37:31.219 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.219+0000 7fc631b6a640 1 -- 172.21.15.163:0/496042332 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc6180762d0 msgr2=0x7fc618078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.219 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.219+0000 7fc631b6a640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc6180762d0 0x7fc618078790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc620033000 tx=0x7fc620009000 comp rx=0 tx=0).stop 2024-05-08T06:37:31.219 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.219+0000 7fc631b6a640 1 -- 172.21.15.163:0/496042332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc62c0b3f60 msgr2=0x7fc62c14cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.219 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.219+0000 7fc631b6a640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc62c0b3f60 0x7fc62c14cac0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00e8a0 tx=0x7fc61c00ed60 comp rx=0 tx=0).stop 2024-05-08T06:37:31.219 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.220+0000 7fc631b6a640 1 -- 172.21.15.163:0/496042332 shutdown_connections 2024-05-08T06:37:31.219 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.220+0000 7fc631b6a640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc6180762d0 0x7fc618078790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.219 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.220+0000 7fc631b6a640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc62c15f500 0x7fc62c15d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.219 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.220+0000 7fc631b6a640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc62c0b7140 0x7fc62c151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.220 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.220+0000 7fc631b6a640 1 --2- 172.21.15.163:0/496042332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc62c0b3f60 0x7fc62c14cac0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.220 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.220+0000 7fc631b6a640 1 -- 172.21.15.163:0/496042332 >> 172.21.15.163:0/496042332 conn(0x7fc62c06d610 msgr2=0x7fc62c06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:31.220 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.220+0000 7fc631b6a640 1 -- 172.21.15.163:0/496042332 shutdown_connections 2024-05-08T06:37:31.220 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.220+0000 7fc631b6a640 1 -- 172.21.15.163:0/496042332 wait complete. 2024-05-08T06:37:31.221 INFO:tasks.ceph.mds.a.smithi163.stderr: -3967> 2024-05-08T06:37:27.824+0000 7fdc28b40640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T06:37:31.231 DEBUG:teuthology.orchestra.run.smithi163:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event get summary 2024-05-08T06:37:31.253 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2024-05-08T06:37:31.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.276+0000 7f263bc12280 1 -- start start 2024-05-08T06:37:31.275 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.276+0000 7f263bc12280 1 Processor -- start 2024-05-08T06:37:31.284 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f263bc12280 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563cfaa4c000 0x563cfaa27080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.284 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f263bc12280 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cf9c4fc00 0x563cfaa26b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.285 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f263bc12280 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfaa4c800 0x563cfaa27b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.285 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f263bc12280 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563cfaa52480 con 0x563cfaa4c800 2024-05-08T06:37:31.285 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f263bc12280 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563cfaa52600 con 0x563cf9c4fc00 2024-05-08T06:37:31.285 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f263bc12280 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x563cfaa52780 con 0x563cfaa4c000 2024-05-08T06:37:31.285 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f2639f6c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfaa4c800 0x563cfaa27b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.285 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f2639f6c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfaa4c800 0x563cfaa27b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38724/0 (socket says 172.21.15.163:38724) 2024-05-08T06:37:31.285 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f2639f6c640 1 -- 172.21.15.163:0/2561757765 learned_addr learned my addr 172.21.15.163:0/2561757765 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:31.285 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f263976b640 1 --2- 172.21.15.163:0/2561757765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563cfaa4c000 0x563cfaa27080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.285 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f2638f6a640 1 --2- 172.21.15.163:0/2561757765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cf9c4fc00 0x563cfaa26b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.285 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f2639f6c640 1 -- 172.21.15.163:0/2561757765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563cfaa4c000 msgr2=0x563cfaa27080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:31.285 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f2639f6c640 1 --2- 172.21.15.163:0/2561757765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563cfaa4c000 0x563cfaa27080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.286 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f2639f6c640 1 -- 172.21.15.163:0/2561757765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cf9c4fc00 msgr2=0x563cfaa26b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:31.286 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f2639f6c640 1 --2- 172.21.15.163:0/2561757765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cf9c4fc00 0x563cfaa26b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.286 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.277+0000 7f2639f6c640 1 -- 172.21.15.163:0/2561757765 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf9c84780 con 0x563cfaa4c800 2024-05-08T06:37:31.286 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.278+0000 7f2639f6c640 1 --2- 172.21.15.163:0/2561757765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfaa4c800 0x563cfaa27b80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x563cfaa89650 tx=0x563cfa9f00c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a18034594c01b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:31.286 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.278+0000 7f2636f66640 1 -- 172.21.15.163:0/2561757765 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563cf9c74d00 con 0x563cfaa4c800 2024-05-08T06:37:31.286 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.278+0000 7f2636f66640 1 -- 172.21.15.163:0/2561757765 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563cf9c7a700 con 0x563cfaa4c800 2024-05-08T06:37:31.286 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.278+0000 7f2636f66640 1 -- 172.21.15.163:0/2561757765 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563cf9c74680 con 0x563cfaa4c800 2024-05-08T06:37:31.286 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.278+0000 7f263bc12280 1 -- 172.21.15.163:0/2561757765 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf9c84000 con 0x563cfaa4c800 2024-05-08T06:37:31.286 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.278+0000 7f263bc12280 1 -- 172.21.15.163:0/2561757765 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=0}) -- 0x563cf9c85a40 con 0x563cfaa4c800 2024-05-08T06:37:31.286 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.278+0000 7f263bc12280 4 waiting for MDS map... 2024-05-08T06:37:31.286 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.278+0000 7f2636f66640 1 -- 172.21.15.163:0/2561757765 <== mon.0 v2:172.21.15.163:3300/0 4 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x563cfab44600 con 0x563cfaa4c800 2024-05-08T06:37:31.286 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.278+0000 7f2636f66640 1 -- 172.21.15.163:0/2561757765 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 15) ==== 2885+0+0 (secure 0 0 0) 0x563cf9ca9180 con 0x563cfaa4c800 2024-05-08T06:37:31.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.278+0000 7f263bc12280 4 Got MDS map 15 2024-05-08T06:37:31.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.278+0000 7f263bc12280 10 main: JournalTool::main 2024-05-08T06:37:31.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.278+0000 7f263bc12280 4 main: JournalTool: connecting to RADOS... 2024-05-08T06:37:31.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.279+0000 7f263bc12280 1 Processor -- start 2024-05-08T06:37:31.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.279+0000 7f263bc12280 1 -- start start 2024-05-08T06:37:31.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.279+0000 7f263bc12280 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563cfab36c00 0x563cfaa27600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.279+0000 7f263bc12280 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cfab37000 0x563cfaa26580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.279+0000 7f263bc12280 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfab37400 0x563cfaa28100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.279+0000 7f263bc12280 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563cf9cb0180 con 0x563cfab37400 2024-05-08T06:37:31.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.279+0000 7f263bc12280 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563cfaa52f00 con 0x563cfab37000 2024-05-08T06:37:31.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.279+0000 7f263bc12280 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x563cfaa53080 con 0x563cfab36c00 2024-05-08T06:37:31.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.279+0000 7f2638f6a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfab37400 0x563cfaa28100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.279+0000 7f2638f6a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfab37400 0x563cfaa28100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38738/0 (socket says 172.21.15.163:38738) 2024-05-08T06:37:31.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.279+0000 7f2638f6a640 1 -- 172.21.15.163:0/159715626 learned_addr learned my addr 172.21.15.163:0/159715626 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:31.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.279+0000 7f263976b640 1 --2- 172.21.15.163:0/159715626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cfab37000 0x563cfaa26580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f2638f6a640 1 --2- 172.21.15.163:0/159715626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563cfab36c00 0x563cfaa27600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f2638f6a640 1 -- 172.21.15.163:0/159715626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563cfab36c00 msgr2=0x563cfaa27600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:31.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f2638f6a640 1 --2- 172.21.15.163:0/159715626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563cfab36c00 0x563cfaa27600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f2638f6a640 1 -- 172.21.15.163:0/159715626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cfab37000 msgr2=0x563cfaa26580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:31.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f2638f6a640 1 --2- 172.21.15.163:0/159715626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cfab37000 0x563cfaa26580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f2638f6a640 1 -- 172.21.15.163:0/159715626 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf9c85860 con 0x563cfab37400 2024-05-08T06:37:31.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f2638f6a640 1 --2- 172.21.15.163:0/159715626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfab37400 0x563cfaa28100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x563cfab2ea20 tx=0x563cfab4ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293fb2fe05026508 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:31.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f2634f62640 1 -- 172.21.15.163:0/159715626 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563cf9c74d00 con 0x563cfab37400 2024-05-08T06:37:31.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f2634f62640 1 -- 172.21.15.163:0/159715626 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563cf9c7afc0 con 0x563cfab37400 2024-05-08T06:37:31.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f2634f62640 1 -- 172.21.15.163:0/159715626 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563cf9c74680 con 0x563cfab37400 2024-05-08T06:37:31.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f263bc12280 1 -- 172.21.15.163:0/159715626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfab37400 msgr2=0x563cfaa28100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f263bc12280 1 --2- 172.21.15.163:0/159715626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfab37400 0x563cfaa28100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x563cfab2ea20 tx=0x563cfab4ccc0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f263bc12280 1 -- 172.21.15.163:0/159715626 shutdown_connections 2024-05-08T06:37:31.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f263bc12280 1 --2- 172.21.15.163:0/159715626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfab37400 0x563cfaa28100 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f263bc12280 1 --2- 172.21.15.163:0/159715626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cfab37000 0x563cfaa26580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f263bc12280 1 --2- 172.21.15.163:0/159715626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563cfab36c00 0x563cfaa27600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.280+0000 7f263bc12280 1 -- 172.21.15.163:0/159715626 >> 172.21.15.163:0/159715626 conn(0x563cfaa4c400 msgr2=0x563cf9d49800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:31.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.281+0000 7f263bc12280 1 -- 172.21.15.163:0/159715626 shutdown_connections 2024-05-08T06:37:31.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.281+0000 7f263bc12280 1 -- 172.21.15.163:0/159715626 wait complete. 2024-05-08T06:37:31.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.281+0000 7f263bc12280 1 Processor -- start 2024-05-08T06:37:31.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.281+0000 7f263bc12280 1 -- start start 2024-05-08T06:37:31.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263bc12280 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cfab36c00 0x563cfaa29180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263bc12280 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfab37000 0x563cfaa28c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263bc12280 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563cfab37400 0x563cfaa28680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263bc12280 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563cfaa53380 con 0x563cfab37000 2024-05-08T06:37:31.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263bc12280 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563cfaa53200 con 0x563cfab36c00 2024-05-08T06:37:31.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263bc12280 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x563cfaa53500 con 0x563cfab37400 2024-05-08T06:37:31.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263976b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfab37000 0x563cfaa28c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263976b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfab37000 0x563cfaa28c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38740/0 (socket says 172.21.15.163:38740) 2024-05-08T06:37:31.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263976b640 1 -- 172.21.15.163:0/1094528390 learned_addr learned my addr 172.21.15.163:0/1094528390 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:31.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f2638f6a640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563cfab37400 0x563cfaa28680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f2638f6a640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cfab36c00 0x563cfaa29180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263976b640 1 -- 172.21.15.163:0/1094528390 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563cfab37400 msgr2=0x563cfaa28680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263976b640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563cfab37400 0x563cfaa28680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263976b640 1 -- 172.21.15.163:0/1094528390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cfab36c00 msgr2=0x563cfaa29180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263976b640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cfab36c00 0x563cfaa29180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263976b640 1 -- 172.21.15.163:0/1094528390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf9c84780 con 0x563cfab37000 2024-05-08T06:37:31.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f2638f6a640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cfab36c00 0x563cfaa29180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:31.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263976b640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfab37000 0x563cfaa28c00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x563cfab2f6e0 tx=0x563cfa9eec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:31.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263375f640 1 -- 172.21.15.163:0/1094528390 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563cf9c74b60 con 0x563cfab37000 2024-05-08T06:37:31.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263375f640 1 -- 172.21.15.163:0/1094528390 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563cf9c7ac40 con 0x563cfab37000 2024-05-08T06:37:31.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563cf9c84000 con 0x563cfab37000 2024-05-08T06:37:31.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263375f640 1 -- 172.21.15.163:0/1094528390 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563cf9c74d00 con 0x563cfab37000 2024-05-08T06:37:31.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf9c85680 con 0x563cfab37000 2024-05-08T06:37:31.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.282+0000 7f263bc12280 4 main: JournalTool: resolving pool 4 2024-05-08T06:37:31.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.284+0000 7f263375f640 1 -- 172.21.15.163:0/1094528390 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x563cfabb6c00 con 0x563cfab37000 2024-05-08T06:37:31.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.284+0000 7f263375f640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x563cfabaec00 0x563cfaa29700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.284+0000 7f263375f640 1 -- 172.21.15.163:0/1094528390 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x563cfabb4a00 con 0x563cfab37000 2024-05-08T06:37:31.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.284+0000 7f263bc12280 4 main: JournalTool: creating IoCtx.. 2024-05-08T06:37:31.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.284+0000 7f263bc12280 4 main: Executing for rank 0 2024-05-08T06:37:31.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.284+0000 7f2638f6a640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x563cfabaec00 0x563cfaa29700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.284+0000 7f263bc12280 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x563cfab37800 0x563cfaa28100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.284+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 --> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] -- osd_op(unknown.0.0:1 4.f 4:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563cfab37c00 con 0x563cfab37800 2024-05-08T06:37:31.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.284+0000 7f2638f6a640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x563cfab37800 0x563cfaa28100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.284+0000 7f2638f6a640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x563cfabaec00 0x563cfaa29700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x563cfab2eb70 tx=0x563cfa9ee600 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:31.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.285+0000 7f2638f6a640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x563cfab37800 0x563cfaa28100 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:31.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.287+0000 7f2638f6a640 1 -- 172.21.15.163:0/1094528390 <== osd.1 v2:172.21.15.163:6824/2322245857 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x563cf9d5c000 con 0x563cfab37800 2024-05-08T06:37:31.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.287+0000 7f263bc12280 4 Pointer 400.00000000 is readable 2024-05-08T06:37:31.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.287+0000 7f263bc12280 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-08T06:37:31.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.287+0000 7f263bc12280 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x563cfabaf000 0x563cfaa26580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.287+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 --> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] -- osd_op(unknown.0.0:2 4.14 4:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563cfabaf400 con 0x563cfabaf000 2024-05-08T06:37:31.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.287+0000 7f2639f6c640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x563cfabaf000 0x563cfaa26580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.287+0000 7f2639f6c640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x563cfabaf000 0x563cfaa26580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:31.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.287+0000 7f2639f6c640 1 -- 172.21.15.163:0/1094528390 <== osd.3 v2:172.21.15.163:6816/127592521 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv18 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x563cf9d5c500 con 0x563cfabaf000 2024-05-08T06:37:31.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.287+0000 7f263bc12280 10 Header 0x400000 0x402c08 0x4033f6 2024-05-08T06:37:31.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.287+0000 7f263bc12280 10 Starting journal scan from offset 0x402c08 2024-05-08T06:37:31.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.287+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 --> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] -- osd_op(unknown.0.0:3 4.14 4:2e2fa760:::200.00000001:head [read 11272~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563cfabaf800 con 0x563cfabaf000 2024-05-08T06:37:31.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.288+0000 7f2639f6c640 1 -- 172.21.15.163:0/1094528390 <== osd.3 v2:172.21.15.163:6816/127592521 2 ==== osd_op_reply(3 200.00000001 [read 11272~2030 out=2030b] v0'0 uv15 ondisk = 0) ==== 156+0+2030 (crc 0 0 0) 0x563cf9d5c500 con 0x563cfabaf000 2024-05-08T06:37:31.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.288+0000 7f263bc12280 4 Read 0x7ee bytes from 200.00000001 gap=0 2024-05-08T06:37:31.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.288+0000 7f263bc12280 10 Parsing data, 0x7ee bytes available 2024-05-08T06:37:31.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.288+0000 7f263bc12280 10 Attempting decode at 0x402c08 2024-05-08T06:37:31.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.288+0000 7f263bc12280 10 Consumed 0x37c bytes 2024-05-08T06:37:31.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.288+0000 7f263bc12280 10 Valid entry at 0x402c08 2024-05-08T06:37:31.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.288+0000 7f263bc12280 10 Attempting decode at 0x402f84 2024-05-08T06:37:31.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.288+0000 7f263bc12280 10 Consumed 0x472 bytes 2024-05-08T06:37:31.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.288+0000 7f263bc12280 10 Valid entry at 0x402f84 2024-05-08T06:37:31.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.288+0000 7f263bc12280 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x563cfabafc00 0x563cfaa27600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.296 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.288+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 --> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] -- osd_op(unknown.0.0:4 4.1b 4:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x563cfa008000 con 0x563cfabafc00 2024-05-08T06:37:31.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.288+0000 7f263976b640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x563cfabafc00 0x563cfaa27600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.288+0000 7f263976b640 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x563cfabafc00 0x563cfaa27600 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:31.297 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263976b640 1 -- 172.21.15.163:0/1094528390 <== osd.4 v2:172.21.15.192:6824/272496966 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x563cf9d5cf00 con 0x563cfabafc00 2024-05-08T06:37:31.297 INFO:teuthology.orchestra.run.smithi163.stdout:Events by type: 2024-05-08T06:37:31.297 INFO:teuthology.orchestra.run.smithi163.stdout: SUBTREEMAP: 1 2024-05-08T06:37:31.297 INFO:teuthology.orchestra.run.smithi163.stdout: UPDATE: 1 2024-05-08T06:37:31.297 INFO:teuthology.orchestra.run.smithi163.stdout:Errors: 0 2024-05-08T06:37:31.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 4 Reached end of journal objects 2024-05-08T06:37:31.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 4 Scanned objects, 0 missing, 1 valid 2024-05-08T06:37:31.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 4 Events scanned, 0 gaps 2024-05-08T06:37:31.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 4 Found 2 valid events 2024-05-08T06:37:31.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 4 Selected 2 events events for processing 2024-05-08T06:37:31.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 4 2 events 2024-05-08T06:37:31.300 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x563cfab37800 msgr2=0x563cfaa28100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.301 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x563cfab37800 0x563cfaa28100 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.301 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x563cfabaf000 msgr2=0x563cfaa26580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.301 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x563cfabaf000 0x563cfaa26580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.301 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x563cfabafc00 msgr2=0x563cfaa27600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.301 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x563cfabafc00 0x563cfaa27600 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.301 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263976b640 1 -- 172.21.15.163:0/1094528390 reap_dead start 2024-05-08T06:37:31.301 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x563cfabaec00 msgr2=0x563cfaa29700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.301 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x563cfabaec00 0x563cfaa29700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x563cfab2eb70 tx=0x563cfa9ee600 comp rx=0 tx=0).stop 2024-05-08T06:37:31.301 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfab37000 msgr2=0x563cfaa28c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.301 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfab37000 0x563cfaa28c00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x563cfab2f6e0 tx=0x563cfa9eec00 comp rx=0 tx=0).stop 2024-05-08T06:37:31.302 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 shutdown_connections 2024-05-08T06:37:31.302 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x563cfabaec00 0x563cfaa29700 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.302 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 --2- 172.21.15.163:0/1094528390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfab37000 0x563cfaa28c00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.302 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 >> 172.21.15.163:0/1094528390 conn(0x563cfaa4c400 msgr2=0x563cf9d49800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:31.302 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 shutdown_connections 2024-05-08T06:37:31.302 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 -- 172.21.15.163:0/1094528390 wait complete. 2024-05-08T06:37:31.302 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 -- 172.21.15.163:0/2561757765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfaa4c800 msgr2=0x563cfaa27b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.302 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 --2- 172.21.15.163:0/2561757765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfaa4c800 0x563cfaa27b80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x563cfaa89650 tx=0x563cfa9f00c0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.302 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 -- 172.21.15.163:0/2561757765 shutdown_connections 2024-05-08T06:37:31.302 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 --2- 172.21.15.163:0/2561757765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563cfaa4c800 0x563cfaa27b80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.302 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 --2- 172.21.15.163:0/2561757765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563cf9c4fc00 0x563cfaa26b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.303 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 --2- 172.21.15.163:0/2561757765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563cfaa4c000 0x563cfaa27080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.303 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.291+0000 7f263bc12280 1 -- 172.21.15.163:0/2561757765 >> 172.21.15.163:0/2561757765 conn(0x563cf9c4f000 msgr2=0x563cf9d49000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:31.303 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.292+0000 7f263bc12280 1 -- 172.21.15.163:0/2561757765 shutdown_connections 2024-05-08T06:37:31.303 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.292+0000 7f263bc12280 1 -- 172.21.15.163:0/2561757765 wait complete. 2024-05-08T06:37:31.303 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'get', 'summary'] in time 0:00:00.071785, result: Events by type: SUBTREEMAP: 1 UPDATE: 1 Errors: 0 2024-05-08T06:37:31.303 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-08T06:37:31.444 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.445+0000 7fe6d389c640 1 Processor -- start 2024-05-08T06:37:31.444 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.445+0000 7fe6d389c640 1 -- start start 2024-05-08T06:37:31.448 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6d389c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe6cc0b6790 0x7fe6cc0b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.448 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6d389c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6cc0b74c0 con 0x7fe6cc15f220 2024-05-08T06:37:31.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6d389c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6cc0b77a0 con 0x7fe6cc0cfaf0 2024-05-08T06:37:31.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6d389c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe6cc15ea20 con 0x7fe6cc0b6790 2024-05-08T06:37:31.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6d1e39640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe6cc15f220 0x7fe6cc15f670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38380/0 (socket says 172.21.15.163:38380) 2024-05-08T06:37:31.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6d1e39640 1 -- 172.21.15.163:0/2273943063 learned_addr learned my addr 172.21.15.163:0/2273943063 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:31.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6d0e37640 1 --2- 172.21.15.163:0/2273943063 >> v2:172.21.15.192:3300/0 conn(0x7fe6cc0b6790 0x7fe6cc0b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6c3fff640 1 -- 172.21.15.163:0/2273943063 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 591561166 0 0) 0x7fe6cc0b74c0 con 0x7fe6cc15f220 2024-05-08T06:37:31.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6c3fff640 1 -- 172.21.15.163:0/2273943063 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6b8003520 con 0x7fe6cc15f220 2024-05-08T06:37:31.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6c3fff640 1 -- 172.21.15.163:0/2273943063 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2261654185 0 0) 0x7fe6b8003520 con 0x7fe6cc15f220 2024-05-08T06:37:31.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6c3fff640 1 -- 172.21.15.163:0/2273943063 >> v2:172.21.15.192:3300/0 conn(0x7fe6cc0b6790 msgr2=0x7fe6cc0b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:31.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6c3fff640 1 --2- 172.21.15.163:0/2273943063 >> v2:172.21.15.192:3300/0 conn(0x7fe6cc0b6790 0x7fe6cc0b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6c3fff640 1 -- 172.21.15.163:0/2273943063 >> v1:172.21.15.192:6789/0 conn(0x7fe6cc0cfaf0 legacy=0x7fe6cc0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6c3fff640 1 -- 172.21.15.163:0/2273943063 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc15ed00 con 0x7fe6cc15f220 2024-05-08T06:37:31.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6c3fff640 1 -- 172.21.15.163:0/2273943063 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7fe6c4001a70 con 0x7fe6cc15f220 2024-05-08T06:37:31.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6c3fff640 1 -- 172.21.15.163:0/2273943063 >> v1:172.21.15.163:6789/0 conn(0x7fe6cc15f220 legacy=0x7fe6cc15f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.446+0000 7fe6c3fff640 1 --2- 172.21.15.163:0/2273943063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b8003a80 0x7fe6b8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6c3fff640 1 --2- 172.21.15.163:0/2273943063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6b8004800 0x7fe6b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6c3fff640 1 --2- 172.21.15.163:0/2273943063 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b80052e0 0x7fe6b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6c3fff640 1 -- 172.21.15.163:0/2273943063 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc15ea20 con 0x7fe6b8004800 2024-05-08T06:37:31.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6c3fff640 1 -- 172.21.15.163:0/2273943063 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b800c1e0 con 0x7fe6b8003a80 2024-05-08T06:37:31.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6c3fff640 1 -- 172.21.15.163:0/2273943063 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b800c3e0 con 0x7fe6b80052e0 2024-05-08T06:37:31.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6d0e37640 1 --2- 172.21.15.163:0/2273943063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6b8004800 0x7fe6b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6d1638640 1 --2- 172.21.15.163:0/2273943063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b8003a80 0x7fe6b8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6d1e39640 1 --2- 172.21.15.163:0/2273943063 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b80052e0 0x7fe6b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6d0e37640 1 -- 172.21.15.163:0/2273943063 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b80052e0 msgr2=0x7fe6b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:31.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6d0e37640 1 --2- 172.21.15.163:0/2273943063 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b80052e0 0x7fe6b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6d0e37640 1 -- 172.21.15.163:0/2273943063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b8003a80 msgr2=0x7fe6b8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:31.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6d0e37640 1 --2- 172.21.15.163:0/2273943063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b8003a80 0x7fe6b8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6d0e37640 1 -- 172.21.15.163:0/2273943063 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b8003520 con 0x7fe6b8004800 2024-05-08T06:37:31.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6d1e39640 1 --2- 172.21.15.163:0/2273943063 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b80052e0 0x7fe6b800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:31.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.447+0000 7fe6d1e39640 1 -- 172.21.15.163:0/2273943063 reap_dead start 2024-05-08T06:37:31.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.448+0000 7fe6d0e37640 1 --2- 172.21.15.163:0/2273943063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6b8004800 0x7fe6b8004c80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009a90 tx=0x7fe6b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5c6c32f4161fee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:31.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.448+0000 7fe6c3fff640 1 -- 172.21.15.163:0/2273943063 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc036070 con 0x7fe6b8004800 2024-05-08T06:37:31.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.448+0000 7fe6c3fff640 1 -- 172.21.15.163:0/2273943063 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6bc031400 con 0x7fe6b8004800 2024-05-08T06:37:31.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.448+0000 7fe6c3fff640 1 -- 172.21.15.163:0/2273943063 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc009690 con 0x7fe6b8004800 2024-05-08T06:37:31.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.448+0000 7fe6d389c640 1 -- 172.21.15.163:0/2273943063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6b8004800 msgr2=0x7fe6b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.448+0000 7fe6d389c640 1 --2- 172.21.15.163:0/2273943063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6b8004800 0x7fe6b8004c80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009a90 tx=0x7fe6b8005830 comp rx=0 tx=0).stop 2024-05-08T06:37:31.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.448+0000 7fe6d389c640 1 -- 172.21.15.163:0/2273943063 shutdown_connections 2024-05-08T06:37:31.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.448+0000 7fe6d389c640 1 --2- 172.21.15.163:0/2273943063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6b8004800 0x7fe6b8004c80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.448+0000 7fe6d389c640 1 -- 172.21.15.163:0/2273943063 >> 172.21.15.163:0/2273943063 conn(0x7fe6cc0706c0 msgr2=0x7fe6cc070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:31.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.448+0000 7fe6d389c640 1 -- 172.21.15.163:0/2273943063 shutdown_connections 2024-05-08T06:37:31.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.448+0000 7fe6d389c640 1 -- 172.21.15.163:0/2273943063 wait complete. 2024-05-08T06:37:31.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.450+0000 7fe6d389c640 1 Processor -- start 2024-05-08T06:37:31.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.450+0000 7fe6d389c640 1 -- start start 2024-05-08T06:37:31.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.450+0000 7fe6d389c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b8004800 0x7fe6cc1f9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.450+0000 7fe6d389c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6cc0ccda0 0x7fe6cc1f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.450+0000 7fe6d389c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6cc1f8150 0x7fe6cc071e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.450+0000 7fe6d389c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc0d0050 con 0x7fe6cc1f8150 2024-05-08T06:37:31.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.450+0000 7fe6d389c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc0cfed0 con 0x7fe6b8004800 2024-05-08T06:37:31.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.450+0000 7fe6d389c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe6cc0d01d0 con 0x7fe6cc0ccda0 2024-05-08T06:37:31.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.450+0000 7fe6d1e39640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6cc1f8150 0x7fe6cc071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.450+0000 7fe6d1e39640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6cc1f8150 0x7fe6cc071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38750/0 (socket says 172.21.15.163:38750) 2024-05-08T06:37:31.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.450+0000 7fe6d1e39640 1 -- 172.21.15.163:0/375674924 learned_addr learned my addr 172.21.15.163:0/375674924 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:31.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.451+0000 7fe6d1e39640 1 -- 172.21.15.163:0/375674924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6cc0ccda0 msgr2=0x7fe6cc1f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.451+0000 7fe6d1e39640 1 --2- 172.21.15.163:0/375674924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6cc0ccda0 0x7fe6cc1f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.451+0000 7fe6d1e39640 1 -- 172.21.15.163:0/375674924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b8004800 msgr2=0x7fe6cc1f9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:31.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.451+0000 7fe6d1e39640 1 --2- 172.21.15.163:0/375674924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b8004800 0x7fe6cc1f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:31.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.451+0000 7fe6d1e39640 1 -- 172.21.15.163:0/375674924 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc1f98c0 con 0x7fe6cc1f8150 2024-05-08T06:37:31.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.451+0000 7fe6d1e39640 1 --2- 172.21.15.163:0/375674924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6cc1f8150 0x7fe6cc071e20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0b5eb0 tx=0x7fe6c4007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:31.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.451+0000 7fe6c1ffb640 1 -- 172.21.15.163:0/375674924 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400ea20 con 0x7fe6cc1f8150 2024-05-08T06:37:31.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.451+0000 7fe6c1ffb640 1 -- 172.21.15.163:0/375674924 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c400ebe0 con 0x7fe6cc1f8150 2024-05-08T06:37:31.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.451+0000 7fe6d389c640 1 -- 172.21.15.163:0/375674924 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6cc1f6a80 con 0x7fe6cc1f8150 2024-05-08T06:37:31.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.451+0000 7fe6c1ffb640 1 -- 172.21.15.163:0/375674924 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4014430 con 0x7fe6cc1f8150 2024-05-08T06:37:31.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.451+0000 7fe6d389c640 1 -- 172.21.15.163:0/375674924 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6cc0ced80 con 0x7fe6cc1f8150 2024-05-08T06:37:31.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.452+0000 7fe6d389c640 1 -- 172.21.15.163:0/375674924 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6cc06a550 con 0x7fe6cc1f8150 2024-05-08T06:37:31.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.453+0000 7fe6c1ffb640 1 -- 172.21.15.163:0/375674924 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe6c40145d0 con 0x7fe6cc1f8150 2024-05-08T06:37:31.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.453+0000 7fe6c1ffb640 1 --2- 172.21.15.163:0/375674924 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe6b0076300 0x7fe6b00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:31.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.453+0000 7fe6d1638640 1 --2- 172.21.15.163:0/375674924 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe6b0076300 0x7fe6b00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:31.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.453+0000 7fe6c1ffb640 1 -- 172.21.15.163:0/375674924 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7fe6c4097330 con 0x7fe6cc1f8150 2024-05-08T06:37:31.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.454+0000 7fe6d1638640 1 --2- 172.21.15.163:0/375674924 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe6b0076300 0x7fe6b00787c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80017d0 tx=0x7fe6c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:31.457 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.458+0000 7fe6c1ffb640 1 -- 172.21.15.163:0/375674924 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe6cc1f98c0 con 0x7fe6cc1f8150 2024-05-08T06:37:31.673 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:31.674+0000 7fe6d389c640 1 -- 172.21.15.163:0/375674924 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fe6cc0b61a0 con 0x7fe6cc1f8150 2024-05-08T06:37:32.217 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.218+0000 7fe6c1ffb640 1 -- 172.21.15.163:0/375674924 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v16) ==== 162+0+0 (secure 0 0 0) 0x7fe6c4060b90 con 0x7fe6cc1f8150 2024-05-08T06:37:32.218 INFO:teuthology.orchestra.run.smithi163.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-08T06:37:32.220 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.221+0000 7fe6d389c640 1 -- 172.21.15.163:0/375674924 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe6b0076300 msgr2=0x7fe6b00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:32.220 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.221+0000 7fe6d389c640 1 --2- 172.21.15.163:0/375674924 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe6b0076300 0x7fe6b00787c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80017d0 tx=0x7fe6c8009040 comp rx=0 tx=0).stop 2024-05-08T06:37:32.220 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.221+0000 7fe6d389c640 1 -- 172.21.15.163:0/375674924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6cc1f8150 msgr2=0x7fe6cc071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:32.220 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.221+0000 7fe6d389c640 1 --2- 172.21.15.163:0/375674924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6cc1f8150 0x7fe6cc071e20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0b5eb0 tx=0x7fe6c4007bc0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.220 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.221+0000 7fe6d389c640 1 -- 172.21.15.163:0/375674924 shutdown_connections 2024-05-08T06:37:32.221 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.221+0000 7fe6d389c640 1 --2- 172.21.15.163:0/375674924 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe6b0076300 0x7fe6b00787c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.221 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.221+0000 7fe6d389c640 1 --2- 172.21.15.163:0/375674924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6cc1f8150 0x7fe6cc071e20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.221 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.221+0000 7fe6d389c640 1 --2- 172.21.15.163:0/375674924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6cc0ccda0 0x7fe6cc1f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.221 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.221+0000 7fe6d389c640 1 --2- 172.21.15.163:0/375674924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b8004800 0x7fe6cc1f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.221 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.221+0000 7fe6d389c640 1 -- 172.21.15.163:0/375674924 >> 172.21.15.163:0/375674924 conn(0x7fe6cc0706c0 msgr2=0x7fe6cc0b6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:32.221 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.221+0000 7fe6d389c640 1 -- 172.21.15.163:0/375674924 shutdown_connections 2024-05-08T06:37:32.221 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.221+0000 7fe6d389c640 1 -- 172.21.15.163:0/375674924 wait complete. 2024-05-08T06:37:32.236 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:32.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.339+0000 7f81d46e1640 1 Processor -- start 2024-05-08T06:37:32.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.339+0000 7f81d46e1640 1 -- start start 2024-05-08T06:37:32.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.340+0000 7f81d46e1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81cc15f760 0x7f81cc0b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:32.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.340+0000 7f81d46e1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f81cc0d0a00 0x7f81cc0b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:32.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.340+0000 7f81d46e1640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81cc0d0070 con 0x7f81cc0cfc10 2024-05-08T06:37:32.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.340+0000 7f81d46e1640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f81cc15fb40 con 0x7f81cc0d0a00 2024-05-08T06:37:32.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.340+0000 7f81d46e1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f81cc0d36f0 con 0x7f81cc15f760 2024-05-08T06:37:32.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.341+0000 7f81d2c7e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f81cc0d0a00 0x7f81cc0b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:32.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.341+0000 7f81d2c7e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f81cc0d0a00 0x7f81cc0b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38754/0 (socket says 172.21.15.163:38754) 2024-05-08T06:37:32.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.341+0000 7f81d2c7e640 1 -- 172.21.15.163:0/2812024189 learned_addr learned my addr 172.21.15.163:0/2812024189 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:32.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.341+0000 7f81d2c7e640 1 -- 172.21.15.163:0/2812024189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81cc15f760 msgr2=0x7f81cc0b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:32.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.341+0000 7f81d2c7e640 1 --2- 172.21.15.163:0/2812024189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81cc15f760 0x7f81cc0b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.341+0000 7f81d2c7e640 1 -- 172.21.15.163:0/2812024189 >> v1:172.21.15.192:6789/0 conn(0x7f81cc0cfc10 legacy=0x7f81cc0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:32.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.341+0000 7f81d2c7e640 1 -- 172.21.15.163:0/2812024189 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc0d39d0 con 0x7f81cc0d0a00 2024-05-08T06:37:32.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.342+0000 7f81d2c7e640 1 --2- 172.21.15.163:0/2812024189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81cc0d0a00 0x7f81cc0b8710 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f81c4009e50 tx=0x7f81c402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b70cf6488f5dd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:32.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.342+0000 7f81d147b640 1 -- 172.21.15.163:0/2812024189 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4036070 con 0x7f81cc0d0a00 2024-05-08T06:37:32.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.342+0000 7f81d147b640 1 -- 172.21.15.163:0/2812024189 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c4031600 con 0x7f81cc0d0a00 2024-05-08T06:37:32.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.342+0000 7f81d147b640 1 -- 172.21.15.163:0/2812024189 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c40319d0 con 0x7f81cc0d0a00 2024-05-08T06:37:32.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.342+0000 7f81d46e1640 1 -- 172.21.15.163:0/2812024189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81cc0d0a00 msgr2=0x7f81cc0b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:32.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.342+0000 7f81d46e1640 1 --2- 172.21.15.163:0/2812024189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81cc0d0a00 0x7f81cc0b8710 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f81c4009e50 tx=0x7f81c402a5f0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.342+0000 7f81d46e1640 1 -- 172.21.15.163:0/2812024189 shutdown_connections 2024-05-08T06:37:32.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.342+0000 7f81d46e1640 1 --2- 172.21.15.163:0/2812024189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81cc0d0a00 0x7f81cc0b8710 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.342+0000 7f81d46e1640 1 --2- 172.21.15.163:0/2812024189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81cc15f760 0x7f81cc0b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.342+0000 7f81d46e1640 1 -- 172.21.15.163:0/2812024189 >> 172.21.15.163:0/2812024189 conn(0x7f81cc0740e0 msgr2=0x7f81cc06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:32.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.343+0000 7f81d46e1640 1 -- 172.21.15.163:0/2812024189 shutdown_connections 2024-05-08T06:37:32.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.343+0000 7f81d46e1640 1 -- 172.21.15.163:0/2812024189 wait complete. 2024-05-08T06:37:32.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.344+0000 7f81d46e1640 1 Processor -- start 2024-05-08T06:37:32.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.344+0000 7f81d46e1640 1 -- start start 2024-05-08T06:37:32.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.344+0000 7f81d46e1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81cc0cfc10 0x7f81cc0b7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:32.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.344+0000 7f81d46e1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81cc0d0a00 0x7f81cc14f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.344+0000 7f81d46e1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81cc15f760 0x7f81cc150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.344+0000 7f81d46e1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc0d4270 con 0x7f81cc15f760 2024-05-08T06:37:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.344+0000 7f81d46e1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc0d3f90 con 0x7f81cc0d0a00 2024-05-08T06:37:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.344+0000 7f81d46e1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f81cc0d4790 con 0x7f81cc0cfc10 2024-05-08T06:37:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81d2c7e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81cc15f760 0x7f81cc150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81d2c7e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81cc15f760 0x7f81cc150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38762/0 (socket says 172.21.15.163:38762) 2024-05-08T06:37:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81d2c7e640 1 -- 172.21.15.163:0/2180752847 learned_addr learned my addr 172.21.15.163:0/2180752847 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81d2c7e640 1 -- 172.21.15.163:0/2180752847 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81cc0cfc10 msgr2=0x7f81cc0b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81d1c7c640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81cc0d0a00 0x7f81cc14f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81d247d640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81cc0cfc10 0x7f81cc0b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81d2c7e640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81cc0cfc10 0x7f81cc0b7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81d2c7e640 1 -- 172.21.15.163:0/2180752847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81cc0d0a00 msgr2=0x7f81cc14f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81d2c7e640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81cc0d0a00 0x7f81cc14f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81d2c7e640 1 -- 172.21.15.163:0/2180752847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc154f80 con 0x7f81cc15f760 2024-05-08T06:37:32.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81d1c7c640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81cc0d0a00 0x7f81cc14f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:32.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81d2c7e640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81cc15f760 0x7f81cc150ab0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f81c4034040 tx=0x7f81c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:32.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81c2ffd640 1 -- 172.21.15.163:0/2180752847 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4041020 con 0x7f81cc15f760 2024-05-08T06:37:32.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81c2ffd640 1 -- 172.21.15.163:0/2180752847 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c40077a0 con 0x7f81cc15f760 2024-05-08T06:37:32.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81d46e1640 1 -- 172.21.15.163:0/2180752847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81cc14f970 con 0x7f81cc15f760 2024-05-08T06:37:32.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81c2ffd640 1 -- 172.21.15.163:0/2180752847 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4036040 con 0x7f81cc15f760 2024-05-08T06:37:32.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.345+0000 7f81d46e1640 1 -- 172.21.15.163:0/2180752847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81cc150200 con 0x7f81cc15f760 2024-05-08T06:37:32.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.346+0000 7f81d46e1640 1 -- 172.21.15.163:0/2180752847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81cc06a550 con 0x7f81cc15f760 2024-05-08T06:37:32.348 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.347+0000 7f81c2ffd640 1 -- 172.21.15.163:0/2180752847 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f81c4044020 con 0x7f81cc15f760 2024-05-08T06:37:32.348 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.347+0000 7f81c2ffd640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f81b00762d0 0x7f81b0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:32.348 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.347+0000 7f81c2ffd640 1 -- 172.21.15.163:0/2180752847 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f81c40b8480 con 0x7f81cc15f760 2024-05-08T06:37:32.348 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.347+0000 7f81d247d640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f81b00762d0 0x7f81b0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:32.348 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.348+0000 7f81d247d640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f81b00762d0 0x7f81b0078790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f81cc151bb0 tx=0x7f81c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:32.351 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.352+0000 7f81c2ffd640 1 -- 172.21.15.163:0/2180752847 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f81c404b030 con 0x7f81cc15f760 2024-05-08T06:37:32.567 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.568+0000 7f81d46e1640 1 -- 172.21.15.163:0/2180752847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81cc151f10 con 0x7f81cc15f760 2024-05-08T06:37:32.568 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.568+0000 7f81c2ffd640 1 -- 172.21.15.163:0/2180752847 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f81cc154f80 con 0x7f81cc15f760 2024-05-08T06:37:32.568 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:32.568 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1773142301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1773142301},{"type":"v1","addr":"172.21.15.163:6835","nonce":1773142301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2952930735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2952930735},{"type":"v1","addr":"172.21.15.192:6834","nonce":2952930735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1297249355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1297249355},{"type":"v1","addr":"172.21.15.192:6835","nonce":1297249355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:37:22.365453+0000","modified":"2024-05-08T06:37:32.221762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"a","rank":0,"incarnation":17,"state":"up:replay","state_seq":1,"addr":"172.21.15.163:6837/3423234723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":3423234723},{"type":"v1","addr":"172.21.15.163:6837","nonce":3423234723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-05-08T06:37:32.568 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 17 2024-05-08T06:37:32.570 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.570+0000 7f81d46e1640 1 -- 172.21.15.163:0/2180752847 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f81b00762d0 msgr2=0x7f81b0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:32.570 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.570+0000 7f81d46e1640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f81b00762d0 0x7f81b0078790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f81cc151bb0 tx=0x7f81c8008040 comp rx=0 tx=0).stop 2024-05-08T06:37:32.570 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.570+0000 7f81d46e1640 1 -- 172.21.15.163:0/2180752847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81cc15f760 msgr2=0x7f81cc150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:32.570 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.570+0000 7f81d46e1640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81cc15f760 0x7f81cc150ab0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f81c4034040 tx=0x7f81c4004a40 comp rx=0 tx=0).stop 2024-05-08T06:37:32.570 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.571+0000 7f81d46e1640 1 -- 172.21.15.163:0/2180752847 shutdown_connections 2024-05-08T06:37:32.570 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.571+0000 7f81d46e1640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f81b00762d0 0x7f81b0078790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.570 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.571+0000 7f81d46e1640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81cc15f760 0x7f81cc150ab0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.570 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.571+0000 7f81d46e1640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81cc0d0a00 0x7f81cc14f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.571 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.571+0000 7f81d46e1640 1 --2- 172.21.15.163:0/2180752847 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81cc0cfc10 0x7f81cc0b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:32.571 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.571+0000 7f81d46e1640 1 -- 172.21.15.163:0/2180752847 >> 172.21.15.163:0/2180752847 conn(0x7f81cc0740e0 msgr2=0x7f81cc0ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:32.571 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.571+0000 7f81d46e1640 1 -- 172.21.15.163:0/2180752847 shutdown_connections 2024-05-08T06:37:32.571 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:32.571+0000 7f81d46e1640 1 -- 172.21.15.163:0/2180752847 wait complete. 2024-05-08T06:37:32.983 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T06:37:22.365453+0000', 'modified': '2024-05-08T06:37:32.221762+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 32, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5031}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5031': {'gid': 5031, 'name': 'a', 'rank': 0, 'incarnation': 17, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.163:6837/3423234723', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 3423234723}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 3423234723}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-08T06:37:32.984 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5031:up:replay 2024-05-08T06:37:33.985 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:34.097 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.098+0000 7fe81c3dd640 1 Processor -- start 2024-05-08T06:37:34.097 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.098+0000 7fe81c3dd640 1 -- start start 2024-05-08T06:37:34.100 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.099+0000 7fe81c3dd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81406da40 0x7fe81416a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:34.100 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.099+0000 7fe81c3dd640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8140b0560 con 0x7fe8141493c0 2024-05-08T06:37:34.100 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.099+0000 7fe81c3dd640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8140b0840 con 0x7fe8140705f0 2024-05-08T06:37:34.100 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.099+0000 7fe81c3dd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe8140b0340 con 0x7fe81406da40 2024-05-08T06:37:34.100 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.099+0000 7fe81a179640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe8141493c0 0x7fe8140afc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38388/0 (socket says 172.21.15.163:38388) 2024-05-08T06:37:34.100 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.099+0000 7fe81a179640 1 -- 172.21.15.163:0/920757580 learned_addr learned my addr 172.21.15.163:0/920757580 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:34.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.099+0000 7fe81a97a640 1 --2- 172.21.15.163:0/920757580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81406da40 0x7fe81416a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:34.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.099+0000 7fe819177640 1 -- 172.21.15.163:0/920757580 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2763327897 0 0) 0x7fe8140b0560 con 0x7fe8141493c0 2024-05-08T06:37:34.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.099+0000 7fe819177640 1 -- 172.21.15.163:0/920757580 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe800003520 con 0x7fe8141493c0 2024-05-08T06:37:34.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe819177640 1 -- 172.21.15.163:0/920757580 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1901053647 0 0) 0x7fe800003520 con 0x7fe8141493c0 2024-05-08T06:37:34.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe819177640 1 -- 172.21.15.163:0/920757580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81406da40 msgr2=0x7fe81416a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:34.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe819177640 1 --2- 172.21.15.163:0/920757580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81406da40 0x7fe81416a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:34.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe819177640 1 -- 172.21.15.163:0/920757580 >> v1:172.21.15.192:6789/0 conn(0x7fe8140705f0 legacy=0x7fe81406d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:34.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe819177640 1 -- 172.21.15.163:0/920757580 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8140b0b20 con 0x7fe8141493c0 2024-05-08T06:37:34.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe819177640 1 -- 172.21.15.163:0/920757580 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7fe8100016f0 con 0x7fe8141493c0 2024-05-08T06:37:34.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe819177640 1 -- 172.21.15.163:0/920757580 >> v1:172.21.15.163:6789/0 conn(0x7fe8141493c0 legacy=0x7fe8140afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:34.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe819177640 1 --2- 172.21.15.163:0/920757580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8000039e0 0x7fe800003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:34.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe819177640 1 --2- 172.21.15.163:0/920757580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81406da40 0x7fe800004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:34.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe819177640 1 --2- 172.21.15.163:0/920757580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe800004d40 0x7fe8000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:34.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe819177640 1 -- 172.21.15.163:0/920757580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe8140b0340 con 0x7fe8000039e0 2024-05-08T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe819177640 1 -- 172.21.15.163:0/920757580 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe80000bb50 con 0x7fe800004d40 2024-05-08T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe819177640 1 -- 172.21.15.163:0/920757580 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe80000bd50 con 0x7fe81406da40 2024-05-08T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe81a179640 1 --2- 172.21.15.163:0/920757580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8000039e0 0x7fe800003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe81a97a640 1 --2- 172.21.15.163:0/920757580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe800004d40 0x7fe8000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe819978640 1 --2- 172.21.15.163:0/920757580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81406da40 0x7fe800004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe81a179640 1 -- 172.21.15.163:0/920757580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81406da40 msgr2=0x7fe800004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe81a179640 1 --2- 172.21.15.163:0/920757580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81406da40 0x7fe800004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe81a179640 1 -- 172.21.15.163:0/920757580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe800004d40 msgr2=0x7fe8000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe81a179640 1 --2- 172.21.15.163:0/920757580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe800004d40 0x7fe8000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.100+0000 7fe81a179640 1 -- 172.21.15.163:0/920757580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe800003520 con 0x7fe8000039e0 2024-05-08T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.101+0000 7fe81a179640 1 --2- 172.21.15.163:0/920757580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8000039e0 0x7fe800003dc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe81000c190 tx=0x7fe800005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363fa9dcc253ebd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.101+0000 7fe819177640 1 -- 172.21.15.163:0/920757580 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810007c80 con 0x7fe8000039e0 2024-05-08T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.101+0000 7fe819177640 1 -- 172.21.15.163:0/920757580 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe810005ad0 con 0x7fe8000039e0 2024-05-08T06:37:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.101+0000 7fe819177640 1 -- 172.21.15.163:0/920757580 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810007070 con 0x7fe8000039e0 2024-05-08T06:37:34.105 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.102+0000 7fe81c3dd640 1 -- 172.21.15.163:0/920757580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8000039e0 msgr2=0x7fe800003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:34.105 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.102+0000 7fe81c3dd640 1 --2- 172.21.15.163:0/920757580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8000039e0 0x7fe800003dc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe81000c190 tx=0x7fe800005830 comp rx=0 tx=0).stop 2024-05-08T06:37:34.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.102+0000 7fe81a97a640 1 -- 172.21.15.163:0/920757580 reap_dead start 2024-05-08T06:37:34.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.102+0000 7fe81c3dd640 1 -- 172.21.15.163:0/920757580 shutdown_connections 2024-05-08T06:37:34.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.102+0000 7fe81c3dd640 1 -- 172.21.15.163:0/920757580 >> 172.21.15.163:0/920757580 conn(0x7fe814072ba0 msgr2=0x7fe814072f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:34.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.102+0000 7fe81c3dd640 1 -- 172.21.15.163:0/920757580 shutdown_connections 2024-05-08T06:37:34.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.102+0000 7fe81c3dd640 1 -- 172.21.15.163:0/920757580 wait complete. 2024-05-08T06:37:34.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.103+0000 7fe81c3dd640 1 Processor -- start 2024-05-08T06:37:34.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.103+0000 7fe81c3dd640 1 -- start start 2024-05-08T06:37:34.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81c3dd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8000039e0 0x7fe81413d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:34.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81c3dd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe81415ef30 0x7fe814140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:34.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81c3dd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81414fe70 0x7fe8141502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:34.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81c3dd640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe8140b5cc0 con 0x7fe8000039e0 2024-05-08T06:37:34.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81c3dd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe81415fde0 con 0x7fe81415ef30 2024-05-08T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81c3dd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe8140b5e40 con 0x7fe81414fe70 2024-05-08T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81a179640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8000039e0 0x7fe81413d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81a179640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8000039e0 0x7fe81413d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38776/0 (socket says 172.21.15.163:38776) 2024-05-08T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81a179640 1 -- 172.21.15.163:0/1200611314 learned_addr learned my addr 172.21.15.163:0/1200611314 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81a179640 1 -- 172.21.15.163:0/1200611314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81414fe70 msgr2=0x7fe8141502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81a179640 1 --2- 172.21.15.163:0/1200611314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81414fe70 0x7fe8141502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81a179640 1 -- 172.21.15.163:0/1200611314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe81415ef30 msgr2=0x7fe814140ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe819978640 1 --2- 172.21.15.163:0/1200611314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe81415ef30 0x7fe814140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81a179640 1 --2- 172.21.15.163:0/1200611314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe81415ef30 0x7fe814140ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81a179640 1 -- 172.21.15.163:0/1200611314 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe814160500 con 0x7fe8000039e0 2024-05-08T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81a179640 1 --2- 172.21.15.163:0/1200611314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8000039e0 0x7fe81413d290 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe81002acf0 tx=0x7fe8140b0a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe80affd640 1 -- 172.21.15.163:0/1200611314 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8100014b0 con 0x7fe8000039e0 2024-05-08T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe80affd640 1 -- 172.21.15.163:0/1200611314 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe810001650 con 0x7fe8000039e0 2024-05-08T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe81c3dd640 1 -- 172.21.15.163:0/1200611314 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8141607e0 con 0x7fe8000039e0 2024-05-08T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.104+0000 7fe80affd640 1 -- 172.21.15.163:0/1200611314 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8100049e0 con 0x7fe8000039e0 2024-05-08T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.105+0000 7fe81c3dd640 1 -- 172.21.15.163:0/1200611314 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe814160da0 con 0x7fe8000039e0 2024-05-08T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.105+0000 7fe81c3dd640 1 -- 172.21.15.163:0/1200611314 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe81406a550 con 0x7fe8000039e0 2024-05-08T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.106+0000 7fe80affd640 1 -- 172.21.15.163:0/1200611314 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe810004b80 con 0x7fe8000039e0 2024-05-08T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.107+0000 7fe80affd640 1 --2- 172.21.15.163:0/1200611314 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe7f8076300 0x7fe7f80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.107+0000 7fe819978640 1 --2- 172.21.15.163:0/1200611314 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe7f8076300 0x7fe7f80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.107+0000 7fe80affd640 1 -- 172.21.15.163:0/1200611314 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7fe8100b7690 con 0x7fe8000039e0 2024-05-08T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.107+0000 7fe819978640 1 --2- 172.21.15.163:0/1200611314 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe7f8076300 0x7fe7f80787c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe8141f8870 tx=0x7fe804009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:34.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.111+0000 7fe80affd640 1 -- 172.21.15.163:0/1200611314 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe810081020 con 0x7fe8000039e0 2024-05-08T06:37:34.333 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.334+0000 7fe81c3dd640 1 -- 172.21.15.163:0/1200611314 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8141415f0 con 0x7fe8000039e0 2024-05-08T06:37:34.333 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.334+0000 7fe80affd640 1 -- 172.21.15.163:0/1200611314 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5187 (secure 0 0 0) 0x7fe810080920 con 0x7fe8000039e0 2024-05-08T06:37:34.333 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:34.334 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1773142301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1773142301},{"type":"v1","addr":"172.21.15.163:6835","nonce":1773142301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2952930735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2952930735},{"type":"v1","addr":"172.21.15.192:6834","nonce":2952930735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1297249355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1297249355},{"type":"v1","addr":"172.21.15.192:6835","nonce":1297249355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:37:22.365453+0000","modified":"2024-05-08T06:37:33.230102+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"a","rank":0,"incarnation":17,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.163:6837/3423234723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":3423234723},{"type":"v1","addr":"172.21.15.163:6837","nonce":3423234723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-05-08T06:37:34.334 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 19 2024-05-08T06:37:34.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.336+0000 7fe81c3dd640 1 -- 172.21.15.163:0/1200611314 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe7f8076300 msgr2=0x7fe7f80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:34.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.336+0000 7fe81c3dd640 1 --2- 172.21.15.163:0/1200611314 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe7f8076300 0x7fe7f80787c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe8141f8870 tx=0x7fe804009040 comp rx=0 tx=0).stop 2024-05-08T06:37:34.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.336+0000 7fe81c3dd640 1 -- 172.21.15.163:0/1200611314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8000039e0 msgr2=0x7fe81413d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:34.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.336+0000 7fe81c3dd640 1 --2- 172.21.15.163:0/1200611314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8000039e0 0x7fe81413d290 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe81002acf0 tx=0x7fe8140b0a10 comp rx=0 tx=0).stop 2024-05-08T06:37:34.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.336+0000 7fe81c3dd640 1 -- 172.21.15.163:0/1200611314 shutdown_connections 2024-05-08T06:37:34.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.336+0000 7fe81c3dd640 1 --2- 172.21.15.163:0/1200611314 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe7f8076300 0x7fe7f80787c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:34.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.336+0000 7fe81c3dd640 1 --2- 172.21.15.163:0/1200611314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81414fe70 0x7fe8141502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:34.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.337+0000 7fe81c3dd640 1 --2- 172.21.15.163:0/1200611314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe81415ef30 0x7fe814140ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:34.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.337+0000 7fe81c3dd640 1 --2- 172.21.15.163:0/1200611314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8000039e0 0x7fe81413d290 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:34.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.337+0000 7fe81c3dd640 1 -- 172.21.15.163:0/1200611314 >> 172.21.15.163:0/1200611314 conn(0x7fe814072ba0 msgr2=0x7fe8141485e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:34.336 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.337+0000 7fe81c3dd640 1 -- 172.21.15.163:0/1200611314 shutdown_connections 2024-05-08T06:37:34.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:34.337+0000 7fe81c3dd640 1 -- 172.21.15.163:0/1200611314 wait complete. 2024-05-08T06:37:34.749 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T06:37:22.365453+0000', 'modified': '2024-05-08T06:37:33.230102+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 32, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5031}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5031': {'gid': 5031, 'name': 'a', 'rank': 0, 'incarnation': 17, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.163:6837/3423234723', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 3423234723}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 3423234723}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-08T06:37:34.749 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5031:up:rejoin 2024-05-08T06:37:35.751 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:35.863 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.864+0000 7f1ebdb7f640 1 Processor -- start 2024-05-08T06:37:35.863 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.864+0000 7f1ebdb7f640 1 -- start start 2024-05-08T06:37:35.864 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1ebdb7f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1eb815f760 0x7f1eb80b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:35.864 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1ebdb7f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eb80d0a00 0x7f1eb80b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:35.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1ebdb7f640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eb80d0070 con 0x7f1eb80cfc10 2024-05-08T06:37:35.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1ebdb7f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb815fb40 con 0x7f1eb815f760 2024-05-08T06:37:35.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1ebdb7f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb80d36f0 con 0x7f1eb80d0a00 2024-05-08T06:37:35.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1eb7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eb80d0a00 0x7f1eb80b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:35.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1eb77fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1eb815f760 0x7f1eb80b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:35.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1eb77fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1eb815f760 0x7f1eb80b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:50918/0 (socket says 172.21.15.163:50918) 2024-05-08T06:37:35.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1eb77fe640 1 -- 172.21.15.163:0/845503368 learned_addr learned my addr 172.21.15.163:0/845503368 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:35.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1eb7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eb80d0a00 0x7f1eb80b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.163:55940/0 (socket says 172.21.15.163:55940) 2024-05-08T06:37:35.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1eb6ffd640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f1eb80cfc10 0x7f1eb80d02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:55428/0 (socket says 172.21.15.163:55428) 2024-05-08T06:37:35.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1eb7fff640 1 -- 172.21.15.163:0/845503368 >> v2:172.21.15.192:3300/0 conn(0x7f1eb815f760 msgr2=0x7f1eb80b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:35.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1eb7fff640 1 --2- 172.21.15.163:0/845503368 >> v2:172.21.15.192:3300/0 conn(0x7f1eb815f760 0x7f1eb80b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:35.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1eb7fff640 1 -- 172.21.15.163:0/845503368 >> v1:172.21.15.192:6789/0 conn(0x7f1eb80cfc10 legacy=0x7f1eb80d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:35.865 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1eb7fff640 1 -- 172.21.15.163:0/845503368 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb80d39d0 con 0x7f1eb80d0a00 2024-05-08T06:37:35.866 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.865+0000 7f1eb77fe640 1 --2- 172.21.15.163:0/845503368 >> v2:172.21.15.192:3300/0 conn(0x7f1eb815f760 0x7f1eb80b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:35.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.866+0000 7f1eb7fff640 1 --2- 172.21.15.163:0/845503368 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eb80d0a00 0x7f1eb80b8710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009d90 tx=0x7f1eb002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d8b82f66e585eb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:35.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.866+0000 7f1eb67fc640 1 -- 172.21.15.163:0/845503368 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0031600 con 0x7f1eb80d0a00 2024-05-08T06:37:35.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.866+0000 7f1eb67fc640 1 -- 172.21.15.163:0/845503368 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb00317a0 con 0x7f1eb80d0a00 2024-05-08T06:37:35.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.866+0000 7f1eb67fc640 1 -- 172.21.15.163:0/845503368 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0031b70 con 0x7f1eb80d0a00 2024-05-08T06:37:35.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.867+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/845503368 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eb80d0a00 msgr2=0x7f1eb80b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:35.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.867+0000 7f1ebdb7f640 1 --2- 172.21.15.163:0/845503368 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eb80d0a00 0x7f1eb80b8710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009d90 tx=0x7f1eb002a5d0 comp rx=0 tx=0).stop 2024-05-08T06:37:35.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.867+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/845503368 shutdown_connections 2024-05-08T06:37:35.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.867+0000 7f1ebdb7f640 1 --2- 172.21.15.163:0/845503368 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eb80d0a00 0x7f1eb80b8710 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:35.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.867+0000 7f1ebdb7f640 1 --2- 172.21.15.163:0/845503368 >> v2:172.21.15.192:3300/0 conn(0x7f1eb815f760 0x7f1eb80b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:35.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.867+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/845503368 >> 172.21.15.163:0/845503368 conn(0x7f1eb80740e0 msgr2=0x7f1eb806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:35.867 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.867+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/845503368 shutdown_connections 2024-05-08T06:37:35.868 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.867+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/845503368 wait complete. 2024-05-08T06:37:35.870 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.868+0000 7f1ebdb7f640 1 Processor -- start 2024-05-08T06:37:35.870 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.869+0000 7f1ebdb7f640 1 -- start start 2024-05-08T06:37:35.870 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.869+0000 7f1ebdb7f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1eb80cfc10 0x7f1eb81f8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:35.870 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.869+0000 7f1ebdb7f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eb80d0a00 0x7f1eb81f9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:35.870 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.869+0000 7f1ebdb7f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1eb815f760 0x7f1eb80b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:35.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.869+0000 7f1ebdb7f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb80d4270 con 0x7f1eb815f760 2024-05-08T06:37:35.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.869+0000 7f1ebdb7f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb80d3f90 con 0x7f1eb80cfc10 2024-05-08T06:37:35.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.869+0000 7f1ebdb7f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb80d4790 con 0x7f1eb80d0a00 2024-05-08T06:37:35.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.869+0000 7f1eb7fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1eb815f760 0x7f1eb80b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:35.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.869+0000 7f1eb7fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1eb815f760 0x7f1eb80b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38790/0 (socket says 172.21.15.163:38790) 2024-05-08T06:37:35.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.869+0000 7f1eb7fff640 1 -- 172.21.15.163:0/2603150460 learned_addr learned my addr 172.21.15.163:0/2603150460 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:35.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.870+0000 7f1eb6ffd640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eb80d0a00 0x7f1eb81f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:35.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.870+0000 7f1eb77fe640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1eb80cfc10 0x7f1eb81f8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:35.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.870+0000 7f1eb7fff640 1 -- 172.21.15.163:0/2603150460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eb80d0a00 msgr2=0x7f1eb81f9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:35.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.870+0000 7f1eb7fff640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eb80d0a00 0x7f1eb81f9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:35.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.870+0000 7f1eb7fff640 1 -- 172.21.15.163:0/2603150460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1eb80cfc10 msgr2=0x7f1eb81f8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:35.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.870+0000 7f1eb7fff640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1eb80cfc10 0x7f1eb81f8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:35.872 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.870+0000 7f1eb7fff640 1 -- 172.21.15.163:0/2603150460 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb815afa0 con 0x7f1eb815f760 2024-05-08T06:37:35.872 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.870+0000 7f1eb6ffd640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eb80d0a00 0x7f1eb81f9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:35.872 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.870+0000 7f1eb7fff640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1eb815f760 0x7f1eb80b5180 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009eb0 tx=0x7f1eb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:35.872 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.870+0000 7f1e9ffff640 1 -- 172.21.15.163:0/2603150460 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb003f070 con 0x7f1eb815f760 2024-05-08T06:37:35.872 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.870+0000 7f1e9ffff640 1 -- 172.21.15.163:0/2603150460 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb0030b60 con 0x7f1eb815f760 2024-05-08T06:37:35.872 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.870+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/2603150460 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb81469b0 con 0x7f1eb815f760 2024-05-08T06:37:35.872 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.870+0000 7f1e9ffff640 1 -- 172.21.15.163:0/2603150460 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0009450 con 0x7f1eb815f760 2024-05-08T06:37:35.872 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.870+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/2603150460 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb8146c70 con 0x7f1eb815f760 2024-05-08T06:37:35.872 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.871+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/2603150460 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb80b6190 con 0x7f1eb815f760 2024-05-08T06:37:35.873 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.872+0000 7f1e9ffff640 1 -- 172.21.15.163:0/2603150460 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1eb003b410 con 0x7f1eb815f760 2024-05-08T06:37:35.873 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.872+0000 7f1e9ffff640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1ea40762d0 0x7f1ea4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:35.873 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.872+0000 7f1e9ffff640 1 -- 172.21.15.163:0/2603150460 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f1eb00b8ba0 con 0x7f1eb815f760 2024-05-08T06:37:35.873 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.872+0000 7f1eb77fe640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1ea40762d0 0x7f1ea4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:35.873 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.873+0000 7f1eb77fe640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1ea40762d0 0x7f1ea4078790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80cba30 tx=0x7f1ea8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:35.875 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:35.877+0000 7f1e9ffff640 1 -- 172.21.15.163:0/2603150460 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1eb815afa0 con 0x7f1eb815f760 2024-05-08T06:37:36.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.094+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/2603150460 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1eb806a550 con 0x7f1eb815f760 2024-05-08T06:37:36.093 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.094+0000 7f1e9ffff640 1 -- 172.21.15.163:0/2603150460 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5194 (secure 0 0 0) 0x7f1eb0082530 con 0x7f1eb815f760 2024-05-08T06:37:36.094 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:36.094 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1773142301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1773142301},{"type":"v1","addr":"172.21.15.163:6835","nonce":1773142301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2952930735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2952930735},{"type":"v1","addr":"172.21.15.192:6834","nonce":2952930735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1297249355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1297249355},{"type":"v1","addr":"172.21.15.192:6835","nonce":1297249355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:37:22.365453+0000","modified":"2024-05-08T06:37:35.233488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6837/3423234723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":3423234723},{"type":"v1","addr":"172.21.15.163:6837","nonce":3423234723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-05-08T06:37:36.094 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 20 2024-05-08T06:37:36.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.096+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/2603150460 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1ea40762d0 msgr2=0x7f1ea4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.096+0000 7f1ebdb7f640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1ea40762d0 0x7f1ea4078790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80cba30 tx=0x7f1ea8008040 comp rx=0 tx=0).stop 2024-05-08T06:37:36.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.096+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/2603150460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1eb815f760 msgr2=0x7f1eb80b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.096+0000 7f1ebdb7f640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1eb815f760 0x7f1eb80b5180 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009eb0 tx=0x7f1eb0004a40 comp rx=0 tx=0).stop 2024-05-08T06:37:36.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.096+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/2603150460 shutdown_connections 2024-05-08T06:37:36.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.096+0000 7f1ebdb7f640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1ea40762d0 0x7f1ea4078790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.096+0000 7f1ebdb7f640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1eb815f760 0x7f1eb80b5180 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.096+0000 7f1ebdb7f640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eb80d0a00 0x7f1eb81f9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.096+0000 7f1ebdb7f640 1 --2- 172.21.15.163:0/2603150460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1eb80cfc10 0x7f1eb81f8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.096+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/2603150460 >> 172.21.15.163:0/2603150460 conn(0x7f1eb80740e0 msgr2=0x7f1eb80ce530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:36.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.096+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/2603150460 shutdown_connections 2024-05-08T06:37:36.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.096+0000 7f1ebdb7f640 1 -- 172.21.15.163:0/2603150460 wait complete. 2024-05-08T06:37:36.509 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T06:37:22.365453+0000', 'modified': '2024-05-08T06:37:35.233488+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 32, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5031}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5031': {'gid': 5031, 'name': 'a', 'rank': 0, 'incarnation': 17, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.163:6837/3423234723', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 3423234723}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 3423234723}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5031, 'qdb_cluster': [5031]} 2024-05-08T06:37:36.510 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T06:37:36.510 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T06:37:36.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.613+0000 7fa9d0809640 1 Processor -- start 2024-05-08T06:37:36.613 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.613+0000 7fa9d0809640 1 -- start start 2024-05-08T06:37:36.616 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.614+0000 7fa9d0809640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa9cc0d2250 0x7fa9cc0d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.616 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.614+0000 7fa9d0809640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9cc0b8a60 con 0x7fa9cc0b8240 2024-05-08T06:37:36.616 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.614+0000 7fa9d0809640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9cc0b8d40 con 0x7fa9cc15f350 2024-05-08T06:37:36.616 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.614+0000 7fa9d0809640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa9cc0d2c00 con 0x7fa9cc0d2250 2024-05-08T06:37:36.616 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c9d9b640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa9cc0b8240 0x7fa9cc0d3580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38404/0 (socket says 172.21.15.163:38404) 2024-05-08T06:37:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c9d9b640 1 -- 172.21.15.163:0/721190317 learned_addr learned my addr 172.21.15.163:0/721190317 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 -- 172.21.15.163:0/721190317 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1118529456 0 0) 0x7fa9cc0b8a60 con 0x7fa9cc0b8240 2024-05-08T06:37:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 -- 172.21.15.163:0/721190317 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9b8003520 con 0x7fa9cc0b8240 2024-05-08T06:37:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9cad9d640 1 --2- 172.21.15.163:0/721190317 >> v2:172.21.15.192:3300/0 conn(0x7fa9cc0d2250 0x7fa9cc0d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 -- 172.21.15.163:0/721190317 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1459316302 0 0) 0x7fa9b8003520 con 0x7fa9cc0b8240 2024-05-08T06:37:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 -- 172.21.15.163:0/721190317 >> v2:172.21.15.192:3300/0 conn(0x7fa9cc0d2250 msgr2=0x7fa9cc0d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 --2- 172.21.15.163:0/721190317 >> v2:172.21.15.192:3300/0 conn(0x7fa9cc0d2250 0x7fa9cc0d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 -- 172.21.15.163:0/721190317 >> v1:172.21.15.192:6789/0 conn(0x7fa9cc15f350 legacy=0x7fa9cc0bace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 -- 172.21.15.163:0/721190317 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9cc0b9020 con 0x7fa9cc0b8240 2024-05-08T06:37:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 -- 172.21.15.163:0/721190317 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7fa9b40016f0 con 0x7fa9cc0b8240 2024-05-08T06:37:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 -- 172.21.15.163:0/721190317 >> v1:172.21.15.163:6789/0 conn(0x7fa9cc0b8240 legacy=0x7fa9cc0d3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 --2- 172.21.15.163:0/721190317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b8003a50 0x7fa9b8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 --2- 172.21.15.163:0/721190317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b8004800 0x7fa9b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 --2- 172.21.15.163:0/721190317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9b80052e0 0x7fa9b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 -- 172.21.15.163:0/721190317 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc0d2c00 con 0x7fa9b8003a50 2024-05-08T06:37:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 -- 172.21.15.163:0/721190317 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b800c1a0 con 0x7fa9b80052e0 2024-05-08T06:37:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c959a640 1 -- 172.21.15.163:0/721190317 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b800c3a0 con 0x7fa9b8004800 2024-05-08T06:37:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/721190317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b8003a50 0x7fa9b8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9c9d9b640 1 --2- 172.21.15.163:0/721190317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b8004800 0x7fa9b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.615+0000 7fa9cad9d640 1 --2- 172.21.15.163:0/721190317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9b80052e0 0x7fa9b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.616+0000 7fa9ca59c640 1 -- 172.21.15.163:0/721190317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b8004800 msgr2=0x7fa9b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.616+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/721190317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b8004800 0x7fa9b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.616+0000 7fa9ca59c640 1 -- 172.21.15.163:0/721190317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9b80052e0 msgr2=0x7fa9b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.616+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/721190317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9b80052e0 0x7fa9b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.616+0000 7fa9ca59c640 1 -- 172.21.15.163:0/721190317 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b8003520 con 0x7fa9b8003a50 2024-05-08T06:37:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.616+0000 7fa9cad9d640 1 -- 172.21.15.163:0/721190317 reap_dead start 2024-05-08T06:37:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.616+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/721190317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b8003a50 0x7fa9b8003e70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4001e60 tx=0x7fa9b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e07e7cbf9e727e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.616+0000 7fa9c959a640 1 -- 172.21.15.163:0/721190317 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4001ab0 con 0x7fa9b8003a50 2024-05-08T06:37:36.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.616+0000 7fa9c959a640 1 -- 172.21.15.163:0/721190317 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c402add0 con 0x7fa9b8003a50 2024-05-08T06:37:36.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.616+0000 7fa9c959a640 1 -- 172.21.15.163:0/721190317 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4003b70 con 0x7fa9b8003a50 2024-05-08T06:37:36.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.617+0000 7fa9d0809640 1 -- 172.21.15.163:0/721190317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b8003a50 msgr2=0x7fa9b8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.617+0000 7fa9d0809640 1 --2- 172.21.15.163:0/721190317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b8003a50 0x7fa9b8003e70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4001e60 tx=0x7fa9b8005830 comp rx=0 tx=0).stop 2024-05-08T06:37:36.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.617+0000 7fa9d0809640 1 -- 172.21.15.163:0/721190317 shutdown_connections 2024-05-08T06:37:36.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.617+0000 7fa9d0809640 1 --2- 172.21.15.163:0/721190317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b8003a50 0x7fa9b8003e70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.617+0000 7fa9d0809640 1 -- 172.21.15.163:0/721190317 >> 172.21.15.163:0/721190317 conn(0x7fa9cc073e70 msgr2=0x7fa9cc074280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:36.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.617+0000 7fa9d0809640 1 -- 172.21.15.163:0/721190317 shutdown_connections 2024-05-08T06:37:36.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.617+0000 7fa9d0809640 1 -- 172.21.15.163:0/721190317 wait complete. 2024-05-08T06:37:36.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.618+0000 7fa9d0809640 1 Processor -- start 2024-05-08T06:37:36.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.618+0000 7fa9d0809640 1 -- start start 2024-05-08T06:37:36.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.619+0000 7fa9d0809640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b8003a50 0x7fa9cc161b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.619+0000 7fa9d0809640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc1518e0 0x7fa9cc0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.619+0000 7fa9d0809640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9cc162110 0x7fa9cc0d3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.619+0000 7fa9d0809640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc15f730 con 0x7fa9b8003a50 2024-05-08T06:37:36.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.619+0000 7fa9d0809640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc0b9670 con 0x7fa9cc1518e0 2024-05-08T06:37:36.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.619+0000 7fa9d0809640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa9cc0b86a0 con 0x7fa9cc162110 2024-05-08T06:37:36.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.619+0000 7fa9ca59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b8003a50 0x7fa9cc161b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.619+0000 7fa9ca59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b8003a50 0x7fa9cc161b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38808/0 (socket says 172.21.15.163:38808) 2024-05-08T06:37:36.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.619+0000 7fa9ca59c640 1 -- 172.21.15.163:0/2806957014 learned_addr learned my addr 172.21.15.163:0/2806957014 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:36.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.620+0000 7fa9ca59c640 1 -- 172.21.15.163:0/2806957014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9cc162110 msgr2=0x7fa9cc0d3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.620+0000 7fa9cad9d640 1 --2- 172.21.15.163:0/2806957014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9cc162110 0x7fa9cc0d3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.620+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/2806957014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9cc162110 0x7fa9cc0d3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.620+0000 7fa9ca59c640 1 -- 172.21.15.163:0/2806957014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc1518e0 msgr2=0x7fa9cc0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.620+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/2806957014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc1518e0 0x7fa9cc0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.620+0000 7fa9ca59c640 1 -- 172.21.15.163:0/2806957014 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9cc0cba50 con 0x7fa9b8003a50 2024-05-08T06:37:36.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.620+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/2806957014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b8003a50 0x7fa9cc161b10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4001e60 tx=0x7fa9c40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:36.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.620+0000 7fa9c2ffd640 1 -- 172.21.15.163:0/2806957014 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4040030 con 0x7fa9b8003a50 2024-05-08T06:37:36.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.620+0000 7fa9c2ffd640 1 -- 172.21.15.163:0/2806957014 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c4003e30 con 0x7fa9b8003a50 2024-05-08T06:37:36.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.620+0000 7fa9d0809640 1 -- 172.21.15.163:0/2806957014 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9cc162590 con 0x7fa9b8003a50 2024-05-08T06:37:36.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.620+0000 7fa9c2ffd640 1 -- 172.21.15.163:0/2806957014 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4001ab0 con 0x7fa9b8003a50 2024-05-08T06:37:36.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.620+0000 7fa9d0809640 1 -- 172.21.15.163:0/2806957014 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9cc0cb770 con 0x7fa9b8003a50 2024-05-08T06:37:36.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.622+0000 7fa9c2ffd640 1 -- 172.21.15.163:0/2806957014 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa9c4043070 con 0x7fa9b8003a50 2024-05-08T06:37:36.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.622+0000 7fa9c2ffd640 1 --2- 172.21.15.163:0/2806957014 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa9ac0762d0 0x7fa9ac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.622+0000 7fa9c2ffd640 1 -- 172.21.15.163:0/2806957014 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7fa9c40b7370 con 0x7fa9b8003a50 2024-05-08T06:37:36.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.622+0000 7fa9c9d9b640 1 --2- 172.21.15.163:0/2806957014 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa9ac0762d0 0x7fa9ac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.623+0000 7fa9c9d9b640 1 --2- 172.21.15.163:0/2806957014 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa9ac0762d0 0x7fa9ac078790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4000fb0 tx=0x7fa9b4001c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:36.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.623+0000 7fa9d35ff740 1 Processor -- start 2024-05-08T06:37:36.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.623+0000 7fa9d35ff740 1 -- start start 2024-05-08T06:37:36.630 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9d35ff740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017053ff50 0x560170542760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.630 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9d35ff740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560170540330 0x560170542ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.630 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9d35ff740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x560170512bb0 0x5601705163c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9d35ff740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56017047c8d0 con 0x56017053ff50 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9d35ff740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560170435c10 con 0x560170540330 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9d35ff740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x560170478890 con 0x560170512bb0 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9ca59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017053ff50 0x560170542760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9ca59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017053ff50 0x560170542760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38814/0 (socket says 172.21.15.163:38814) 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9ca59c640 1 -- 172.21.15.163:0/3206690974 learned_addr learned my addr 172.21.15.163:0/3206690974 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9ca59c640 1 -- 172.21.15.163:0/3206690974 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x560170512bb0 msgr2=0x5601705163c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/3206690974 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x560170512bb0 0x5601705163c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9ca59c640 1 -- 172.21.15.163:0/3206690974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560170540330 msgr2=0x560170542ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9c9d9b640 1 --2- 172.21.15.163:0/3206690974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560170540330 0x560170542ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/3206690974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560170540330 0x560170542ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9ca59c640 1 -- 172.21.15.163:0/3206690974 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56017043d5d0 con 0x56017053ff50 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.624+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/3206690974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017053ff50 0x560170542760 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9c407c560 tx=0x7fa9c407fff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb53c34e0347ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.625+0000 7fa9abfff640 1 -- 172.21.15.163:0/3206690974 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c40804e0 con 0x56017053ff50 2024-05-08T06:37:36.631 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.625+0000 7fa9abfff640 1 -- 172.21.15.163:0/3206690974 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c4084890 con 0x56017053ff50 2024-05-08T06:37:36.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.625+0000 7fa9abfff640 1 -- 172.21.15.163:0/3206690974 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4089340 con 0x56017053ff50 2024-05-08T06:37:36.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.625+0000 7fa9d35ff740 1 -- 172.21.15.163:0/3206690974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017053ff50 msgr2=0x560170542760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.625+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/3206690974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017053ff50 0x560170542760 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9c407c560 tx=0x7fa9c407fff0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.625+0000 7fa9d35ff740 1 -- 172.21.15.163:0/3206690974 shutdown_connections 2024-05-08T06:37:36.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.625+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/3206690974 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x560170512bb0 0x5601705163c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.625+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/3206690974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560170540330 0x560170542ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.625+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/3206690974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017053ff50 0x560170542760 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.625+0000 7fa9d35ff740 1 -- 172.21.15.163:0/3206690974 >> 172.21.15.163:0/3206690974 conn(0x560170514e20 msgr2=0x56017055d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:36.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.625+0000 7fa9d35ff740 1 -- 172.21.15.163:0/3206690974 shutdown_connections 2024-05-08T06:37:36.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.625+0000 7fa9d35ff740 1 -- 172.21.15.163:0/3206690974 wait complete. 2024-05-08T06:37:36.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.626+0000 7fa9d35ff740 1 -- start start 2024-05-08T06:37:36.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.626+0000 7fa9d35ff740 1 Processor -- start 2024-05-08T06:37:36.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9d35ff740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56017053ff50 0x560170537170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9d35ff740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560170540330 0x560170559740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9d35ff740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560170558fd0 0x560170511a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9d35ff740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560170435c10 con 0x560170558fd0 2024-05-08T06:37:36.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9d35ff740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560170439760 con 0x560170540330 2024-05-08T06:37:36.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9d35ff740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x56017047c8d0 con 0x56017053ff50 2024-05-08T06:37:36.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9d35ff740 10 client.0 _ll_register_callbacks cb 0x560170472cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:37:36.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9cad9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560170558fd0 0x560170511a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9cad9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560170558fd0 0x560170511a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38824/0 (socket says 172.21.15.163:38824) 2024-05-08T06:37:36.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9cad9d640 1 -- 172.21.15.163:0/3137494219 learned_addr learned my addr 172.21.15.163:0/3137494219 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:36.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/3137494219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56017053ff50 0x560170537170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9c9d9b640 1 --2- 172.21.15.163:0/3137494219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560170540330 0x560170559740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9cad9d640 1 -- 172.21.15.163:0/3137494219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56017053ff50 msgr2=0x560170537170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:36.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9cad9d640 1 --2- 172.21.15.163:0/3137494219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56017053ff50 0x560170537170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.633 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9cad9d640 1 -- 172.21.15.163:0/3137494219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560170540330 msgr2=0x560170559740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:36.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9cad9d640 1 --2- 172.21.15.163:0/3137494219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560170540330 0x560170559740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9cad9d640 1 -- 172.21.15.163:0/3137494219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017018e2d0 con 0x560170558fd0 2024-05-08T06:37:36.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9cad9d640 1 --2- 172.21.15.163:0/3137494219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560170558fd0 0x560170511a40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0c8bd0 tx=0x7fa9bc00e540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa7ac9093c9dcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:36.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.627+0000 7fa9a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:37:36.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9a97fa640 1 -- 172.21.15.163:0/3137494219 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc018070 con 0x560170558fd0 2024-05-08T06:37:36.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9a97fa640 1 -- 172.21.15.163:0/3137494219 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7fa9bc00ed40 con 0x560170558fd0 2024-05-08T06:37:36.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9d35ff740 1 -- 172.21.15.163:0/3137494219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c4084890 con 0x560170558fd0 2024-05-08T06:37:36.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9a97fa640 1 -- 172.21.15.163:0/3137494219 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa9bc023050 con 0x560170558fd0 2024-05-08T06:37:36.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9d35ff740 10 client.5076 fetch_fsmap learned FSMap version 20 2024-05-08T06:37:36.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9d35ff740 1 -- 172.21.15.163:0/3137494219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x5601702586b0 con 0x560170558fd0 2024-05-08T06:37:36.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9a97fa640 1 -- 172.21.15.163:0/3137494219 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c4084890 con 0x560170558fd0 2024-05-08T06:37:36.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9a97fa640 1 -- 172.21.15.163:0/3137494219 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fa9bc026020 con 0x560170558fd0 2024-05-08T06:37:36.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9d35ff740 10 client.5076 fetch_fsmap finished waiting for FSMap version 20 2024-05-08T06:37:36.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9d35ff740 10 client.5076 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T06:37:36.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9d35ff740 20 client.5076 populate_metadata read hostname 'smithi163' 2024-05-08T06:37:36.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/3137494219 >> [v2:172.21.15.163:6836/3423234723,v1:172.21.15.163:6837/3423234723] conn(0x5601705442f0 0x56017055eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9d35ff740 4 client.5076 mds_command: new command op to 5031 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:37:36.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9d35ff740 1 -- 172.21.15.163:0/3137494219 --> [v2:172.21.15.163:6836/3423234723,v1:172.21.15.163:6837/3423234723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560170400760 con 0x5601705442f0 2024-05-08T06:37:36.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/3137494219 >> [v2:172.21.15.163:6836/3423234723,v1:172.21.15.163:6837/3423234723] conn(0x5601705442f0 0x56017055eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9a97fa640 10 client.5076 ms_handle_connect on v2:172.21.15.163:6836/3423234723 2024-05-08T06:37:36.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.628+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/3137494219 >> [v2:172.21.15.163:6836/3423234723,v1:172.21.15.163:6837/3423234723] conn(0x5601705442f0 0x56017055eeb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e80272b8519555 server_cookie=b22f7a824d6cf530 in_seq=0 out_seq=0 2024-05-08T06:37:36.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.629+0000 7fa9a97fa640 1 -- 172.21.15.163:0/3137494219 <== mds.0 v2:172.21.15.163:6836/3423234723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560170400760 con 0x5601705442f0 2024-05-08T06:37:36.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.629+0000 7fa9a97fa640 10 client.5076 handle_command_reply: tid=0 2024-05-08T06:37:36.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.629+0000 7fa9d35ff740 1 client.5076 shutdown 2024-05-08T06:37:36.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.629+0000 7fa9d35ff740 1 -- 172.21.15.163:0/3137494219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560170558fd0 msgr2=0x560170511a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.629+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/3137494219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560170558fd0 0x560170511a40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0c8bd0 tx=0x7fa9bc00e540 comp rx=0 tx=0).stop 2024-05-08T06:37:36.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.629+0000 7fa9d35ff740 1 -- 172.21.15.163:0/3137494219 shutdown_connections 2024-05-08T06:37:36.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.629+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/3137494219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560170558fd0 0x560170511a40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.629+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/3137494219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560170540330 0x560170559740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.629+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/3137494219 >> [v2:172.21.15.163:6836/3423234723,v1:172.21.15.163:6837/3423234723] conn(0x5601705442f0 0x56017055eeb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.629+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/3137494219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56017053ff50 0x560170537170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.629+0000 7fa9a97fa640 0 client.5076 ms_handle_reset on v2:172.21.15.163:6836/3423234723 2024-05-08T06:37:36.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.629+0000 7fa9d35ff740 1 -- 172.21.15.163:0/3137494219 >> 172.21.15.163:0/3137494219 conn(0x560170512bb0 msgr2=0x560170542960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:36.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.629+0000 7fa9d35ff740 1 -- 172.21.15.163:0/3137494219 shutdown_connections 2024-05-08T06:37:36.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.629+0000 7fa9d35ff740 1 -- 172.21.15.163:0/3137494219 wait complete. 2024-05-08T06:37:36.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.630+0000 7fa9d35ff740 20 client.5076 trim_cache size 0 max 16384 2024-05-08T06:37:36.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.648+0000 7fa9d35ff740 1 Processor -- start 2024-05-08T06:37:36.647 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.648+0000 7fa9d35ff740 1 -- start start 2024-05-08T06:37:36.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9d35ff740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017053ff50 0x56017055a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9d35ff740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x560170540330 0x56017055a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.649 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9d35ff740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560170558fd0 0x560170580e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9d35ff740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc2012e0 con 0x56017053ff50 2024-05-08T06:37:36.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9d35ff740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc1f5f10 con 0x560170558fd0 2024-05-08T06:37:36.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9d35ff740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa9cc200c70 con 0x560170540330 2024-05-08T06:37:36.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9ca59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017053ff50 0x56017055a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9ca59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017053ff50 0x56017055a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38834/0 (socket says 172.21.15.163:38834) 2024-05-08T06:37:36.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9ca59c640 1 -- 172.21.15.163:0/4221379862 learned_addr learned my addr 172.21.15.163:0/4221379862 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:36.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9ca59c640 1 -- 172.21.15.163:0/4221379862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x560170540330 msgr2=0x56017055a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:36.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/4221379862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x560170540330 0x56017055a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9ca59c640 1 -- 172.21.15.163:0/4221379862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560170558fd0 msgr2=0x560170580e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:36.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/4221379862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560170558fd0 0x560170580e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9ca59c640 1 -- 172.21.15.163:0/4221379862 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560170517d20 con 0x56017053ff50 2024-05-08T06:37:36.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/4221379862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017053ff50 0x56017055a190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40801e0 tx=0x7fa9c4084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8cf70a192c33a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:36.650 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9ab7fe640 1 -- 172.21.15.163:0/4221379862 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c407c3a0 con 0x56017053ff50 2024-05-08T06:37:36.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9ab7fe640 1 -- 172.21.15.163:0/4221379862 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c4084890 con 0x56017053ff50 2024-05-08T06:37:36.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.649+0000 7fa9ab7fe640 1 -- 172.21.15.163:0/4221379862 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c40804e0 con 0x56017053ff50 2024-05-08T06:37:36.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.650+0000 7fa9d35ff740 1 -- 172.21.15.163:0/4221379862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017053ff50 msgr2=0x56017055a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.650+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/4221379862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017053ff50 0x56017055a190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40801e0 tx=0x7fa9c4084040 comp rx=0 tx=0).stop 2024-05-08T06:37:36.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.650+0000 7fa9d35ff740 1 -- 172.21.15.163:0/4221379862 shutdown_connections 2024-05-08T06:37:36.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.650+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/4221379862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560170558fd0 0x560170580e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.650+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/4221379862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x560170540330 0x56017055a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.650+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/4221379862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017053ff50 0x56017055a190 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.650+0000 7fa9d35ff740 1 -- 172.21.15.163:0/4221379862 >> 172.21.15.163:0/4221379862 conn(0x560170512bb0 msgr2=0x560170536610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:36.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.650+0000 7fa9d35ff740 1 -- 172.21.15.163:0/4221379862 shutdown_connections 2024-05-08T06:37:36.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.650+0000 7fa9d35ff740 1 -- 172.21.15.163:0/4221379862 wait complete. 2024-05-08T06:37:36.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.650+0000 7fa9d35ff740 1 -- start start 2024-05-08T06:37:36.651 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.650+0000 7fa9d35ff740 1 Processor -- start 2024-05-08T06:37:36.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.651+0000 7fa9d35ff740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56017053ff50 0x5601705604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.651+0000 7fa9d35ff740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x560170540330 0x560170542750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.651+0000 7fa9d35ff740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560170558fd0 0x56017055a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.651+0000 7fa9d35ff740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560170544170 con 0x560170558fd0 2024-05-08T06:37:36.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.651+0000 7fa9d35ff740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560170558ab0 con 0x56017053ff50 2024-05-08T06:37:36.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.651+0000 7fa9d35ff740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5601705447c0 con 0x560170540330 2024-05-08T06:37:36.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.651+0000 7fa9d35ff740 10 client.0 _ll_register_callbacks cb 0x7fa9bc006bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:37:36.655 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.651+0000 7fa9cad9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560170558fd0 0x56017055a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.651+0000 7fa9cad9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560170558fd0 0x56017055a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38846/0 (socket says 172.21.15.163:38846) 2024-05-08T06:37:36.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.651+0000 7fa9cad9d640 1 -- 172.21.15.163:0/1383945305 learned_addr learned my addr 172.21.15.163:0/1383945305 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:36.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9cad9d640 1 -- 172.21.15.163:0/1383945305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x560170540330 msgr2=0x560170542750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:36.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9cad9d640 1 --2- 172.21.15.163:0/1383945305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x560170540330 0x560170542750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9cad9d640 1 -- 172.21.15.163:0/1383945305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56017053ff50 msgr2=0x5601705604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:36.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9cad9d640 1 --2- 172.21.15.163:0/1383945305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56017053ff50 0x5601705604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9cad9d640 1 -- 172.21.15.163:0/1383945305 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560170542c90 con 0x560170558fd0 2024-05-08T06:37:36.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9cad9d640 1 --2- 172.21.15.163:0/1383945305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560170558fd0 0x56017055a4e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x560170543bd0 tx=0x7fa9bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9f781a768f9b54 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:36.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:37:36.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9a97fa640 1 -- 172.21.15.163:0/1383945305 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc018040 con 0x560170558fd0 2024-05-08T06:37:36.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9a97fa640 1 -- 172.21.15.163:0/1383945305 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7fa9bc004220 con 0x560170558fd0 2024-05-08T06:37:36.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9d35ff740 1 -- 172.21.15.163:0/1383945305 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c4084890 con 0x560170558fd0 2024-05-08T06:37:36.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9a97fa640 1 -- 172.21.15.163:0/1383945305 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa9bc033050 con 0x560170558fd0 2024-05-08T06:37:36.656 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9d35ff740 10 client.5082 fetch_fsmap learned FSMap version 20 2024-05-08T06:37:36.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9d35ff740 1 -- 172.21.15.163:0/1383945305 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x56017055aa20 con 0x560170558fd0 2024-05-08T06:37:36.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9a97fa640 1 -- 172.21.15.163:0/1383945305 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c4084890 con 0x560170558fd0 2024-05-08T06:37:36.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9a97fa640 1 -- 172.21.15.163:0/1383945305 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fa9bc036020 con 0x560170558fd0 2024-05-08T06:37:36.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9d35ff740 10 client.5082 fetch_fsmap finished waiting for FSMap version 20 2024-05-08T06:37:36.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9d35ff740 10 client.5082 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T06:37:36.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9d35ff740 20 client.5082 populate_metadata read hostname 'smithi163' 2024-05-08T06:37:36.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/1383945305 >> [v2:172.21.15.163:6836/3423234723,v1:172.21.15.163:6837/3423234723] conn(0x560170580220 0x560170580600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9d35ff740 4 client.5082 mds_command: new command op to 5031 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T06:37:36.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9d35ff740 1 -- 172.21.15.163:0/1383945305 --> [v2:172.21.15.163:6836/3423234723,v1:172.21.15.163:6837/3423234723] -- command(tid 0: {"prefix": "status"}) -- 0x56017048a4e0 con 0x560170580220 2024-05-08T06:37:36.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.652+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/1383945305 >> [v2:172.21.15.163:6836/3423234723,v1:172.21.15.163:6837/3423234723] conn(0x560170580220 0x560170580600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9a97fa640 10 client.5082 ms_handle_connect on v2:172.21.15.163:6836/3423234723 2024-05-08T06:37:36.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9ca59c640 1 --2- 172.21.15.163:0/1383945305 >> [v2:172.21.15.163:6836/3423234723,v1:172.21.15.163:6837/3423234723] conn(0x560170580220 0x560170580600 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a105c1ba7dcf21c5 server_cookie=fd5a065fcd4ccdc3 in_seq=0 out_seq=0 2024-05-08T06:37:36.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9a97fa640 1 -- 172.21.15.163:0/1383945305 <== mds.0 v2:172.21.15.163:6836/3423234723 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56017048a4e0 con 0x560170580220 2024-05-08T06:37:36.657 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9a97fa640 10 client.5082 handle_command_reply: tid=0 2024-05-08T06:37:36.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9d35ff740 1 client.5082 shutdown 2024-05-08T06:37:36.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9d35ff740 1 -- 172.21.15.163:0/1383945305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560170558fd0 msgr2=0x56017055a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/1383945305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560170558fd0 0x56017055a4e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x560170543bd0 tx=0x7fa9bc0046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9d35ff740 1 -- 172.21.15.163:0/1383945305 shutdown_connections 2024-05-08T06:37:36.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/1383945305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560170558fd0 0x56017055a4e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/1383945305 >> [v2:172.21.15.163:6836/3423234723,v1:172.21.15.163:6837/3423234723] conn(0x560170580220 0x560170580600 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/1383945305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x560170540330 0x560170542750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9d35ff740 1 --2- 172.21.15.163:0/1383945305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56017053ff50 0x5601705604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9a97fa640 0 client.5082 ms_handle_reset on v2:172.21.15.163:6836/3423234723 2024-05-08T06:37:36.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9d35ff740 1 -- 172.21.15.163:0/1383945305 >> 172.21.15.163:0/1383945305 conn(0x560170512bb0 msgr2=0x560170545ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:36.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9d35ff740 1 -- 172.21.15.163:0/1383945305 shutdown_connections 2024-05-08T06:37:36.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9d35ff740 1 -- 172.21.15.163:0/1383945305 wait complete. 2024-05-08T06:37:36.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.653+0000 7fa9d35ff740 20 client.5082 trim_cache size 0 max 16384 2024-05-08T06:37:36.659 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:37:36.659 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "e639506e-cf71-49c6-ae4d-5af4601c36e0", 2024-05-08T06:37:36.659 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2024-05-08T06:37:36.659 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 5031, 2024-05-08T06:37:36.659 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2024-05-08T06:37:36.659 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2024-05-08T06:37:36.659 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2024-05-08T06:37:36.659 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 4.4275379480000003, 2024-05-08T06:37:36.659 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 20, 2024-05-08T06:37:36.659 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 32, 2024-05-08T06:37:36.659 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 32, 2024-05-08T06:37:36.660 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 5.3977467639999999 2024-05-08T06:37:36.660 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:37:36.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.654+0000 7fa9d0809640 1 -- 172.21.15.163:0/2806957014 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa9ac0762d0 msgr2=0x7fa9ac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.654+0000 7fa9d0809640 1 --2- 172.21.15.163:0/2806957014 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa9ac0762d0 0x7fa9ac078790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4000fb0 tx=0x7fa9b4001c60 comp rx=0 tx=0).stop 2024-05-08T06:37:36.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.654+0000 7fa9d0809640 1 -- 172.21.15.163:0/2806957014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b8003a50 msgr2=0x7fa9cc161b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.654+0000 7fa9d0809640 1 --2- 172.21.15.163:0/2806957014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b8003a50 0x7fa9cc161b10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4001e60 tx=0x7fa9c40014a0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.654+0000 7fa9d0809640 1 -- 172.21.15.163:0/2806957014 shutdown_connections 2024-05-08T06:37:36.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.654+0000 7fa9d0809640 1 --2- 172.21.15.163:0/2806957014 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa9ac0762d0 0x7fa9ac078790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.654+0000 7fa9d0809640 1 --2- 172.21.15.163:0/2806957014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9cc162110 0x7fa9cc0d3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.654+0000 7fa9d0809640 1 --2- 172.21.15.163:0/2806957014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc1518e0 0x7fa9cc0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.654+0000 7fa9d0809640 1 --2- 172.21.15.163:0/2806957014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b8003a50 0x7fa9cc161b10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.654+0000 7fa9d0809640 1 -- 172.21.15.163:0/2806957014 >> 172.21.15.163:0/2806957014 conn(0x7fa9cc073e70 msgr2=0x7fa9cc0b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:36.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.654+0000 7fa9d0809640 1 -- 172.21.15.163:0/2806957014 shutdown_connections 2024-05-08T06:37:36.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.654+0000 7fa9d0809640 1 -- 172.21.15.163:0/2806957014 wait complete. 2024-05-08T06:37:36.665 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:36.765 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.766+0000 7f9807c08640 1 Processor -- start 2024-05-08T06:37:36.765 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.766+0000 7f9807c08640 1 -- start start 2024-05-08T06:37:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.767+0000 7f9807c08640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9800142390 0x7f98000b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.767+0000 7f9807c08640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f98001636d0 0x7f9800142a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.767+0000 7f9807c08640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9800142770 con 0x7f98000b65e0 2024-05-08T06:37:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.767+0000 7f9807c08640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f98000b6a40 con 0x7f98001636d0 2024-05-08T06:37:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.767+0000 7f9807c08640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9800163d30 con 0x7f9800142390 2024-05-08T06:37:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.768+0000 7f98061a5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f98001636d0 0x7f9800142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.768+0000 7f98059a4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9800142390 0x7f98000b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.768+0000 7f98051a3640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f98000b65e0 0x7f9800162fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:55440/0 (socket says 172.21.15.163:55440) 2024-05-08T06:37:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.768+0000 7f98061a5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f98001636d0 0x7f9800142a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:50988/0 (socket says 172.21.15.163:50988) 2024-05-08T06:37:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.768+0000 7f98051a3640 1 -- 172.21.15.163:0/1702187370 learned_addr learned my addr 172.21.15.163:0/1702187370 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.768+0000 7f98059a4640 1 -- 172.21.15.163:0/1702187370 >> v2:172.21.15.192:3300/0 conn(0x7f98001636d0 msgr2=0x7f9800142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.768+0000 7f98059a4640 1 --2- 172.21.15.163:0/1702187370 >> v2:172.21.15.192:3300/0 conn(0x7f98001636d0 0x7f9800142a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.768+0000 7f98059a4640 1 -- 172.21.15.163:0/1702187370 >> v1:172.21.15.192:6789/0 conn(0x7f98000b65e0 legacy=0x7f9800162fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.768+0000 7f98059a4640 1 -- 172.21.15.163:0/1702187370 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980015f990 con 0x7f9800142390 2024-05-08T06:37:36.769 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.768+0000 7f98061a5640 1 --2- 172.21.15.163:0/1702187370 >> v2:172.21.15.192:3300/0 conn(0x7f98001636d0 0x7f9800142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:36.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.769+0000 7f98059a4640 1 --2- 172.21.15.163:0/1702187370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9800142390 0x7f98000b60a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0097a0 tx=0x7f97fc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee7d970d349d1786 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:36.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.769+0000 7f98049a2640 1 -- 172.21.15.163:0/1702187370 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc037070 con 0x7f9800142390 2024-05-08T06:37:36.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.769+0000 7f98049a2640 1 -- 172.21.15.163:0/1702187370 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97fc032440 con 0x7f9800142390 2024-05-08T06:37:36.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.769+0000 7f98049a2640 1 -- 172.21.15.163:0/1702187370 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc032810 con 0x7f9800142390 2024-05-08T06:37:36.771 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.769+0000 7f9807c08640 1 -- 172.21.15.163:0/1702187370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9800142390 msgr2=0x7f98000b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.771 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.770+0000 7f9807c08640 1 --2- 172.21.15.163:0/1702187370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9800142390 0x7f98000b60a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0097a0 tx=0x7f97fc02a180 comp rx=0 tx=0).stop 2024-05-08T06:37:36.771 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.770+0000 7f9807c08640 1 -- 172.21.15.163:0/1702187370 shutdown_connections 2024-05-08T06:37:36.771 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.770+0000 7f9807c08640 1 --2- 172.21.15.163:0/1702187370 >> v2:172.21.15.192:3300/0 conn(0x7f98001636d0 0x7f9800142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.771 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.770+0000 7f9807c08640 1 --2- 172.21.15.163:0/1702187370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9800142390 0x7f98000b60a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.771 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.770+0000 7f9807c08640 1 -- 172.21.15.163:0/1702187370 >> 172.21.15.163:0/1702187370 conn(0x7f98000b4da0 msgr2=0x7f98000b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:36.771 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.770+0000 7f9807c08640 1 -- 172.21.15.163:0/1702187370 shutdown_connections 2024-05-08T06:37:36.771 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.770+0000 7f9807c08640 1 -- 172.21.15.163:0/1702187370 wait complete. 2024-05-08T06:37:36.771 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.771+0000 7f9807c08640 1 Processor -- start 2024-05-08T06:37:36.771 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.771+0000 7f9807c08640 1 -- start start 2024-05-08T06:37:36.773 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f9807c08640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98000b65e0 0x7f98001f8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f9807c08640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9800142390 0x7f98000cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f9807c08640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98001636d0 0x7f9800161ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f9807c08640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9800160230 con 0x7f9800142390 2024-05-08T06:37:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f9807c08640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f980015ff50 con 0x7f98000b65e0 2024-05-08T06:37:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f9807c08640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9800160750 con 0x7f98001636d0 2024-05-08T06:37:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f98051a3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9800142390 0x7f98000cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f98051a3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9800142390 0x7f98000cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38852/0 (socket says 172.21.15.163:38852) 2024-05-08T06:37:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f98051a3640 1 -- 172.21.15.163:0/3937427788 learned_addr learned my addr 172.21.15.163:0/3937427788 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f98059a4640 1 --2- 172.21.15.163:0/3937427788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98000b65e0 0x7f98001f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f98061a5640 1 --2- 172.21.15.163:0/3937427788 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98001636d0 0x7f9800161ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f98051a3640 1 -- 172.21.15.163:0/3937427788 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98001636d0 msgr2=0x7f9800161ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f98051a3640 1 --2- 172.21.15.163:0/3937427788 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98001636d0 0x7f9800161ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f98051a3640 1 -- 172.21.15.163:0/3937427788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98000b65e0 msgr2=0x7f98001f8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f98051a3640 1 --2- 172.21.15.163:0/3937427788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98000b65e0 0x7f98001f8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f98051a3640 1 -- 172.21.15.163:0/3937427788 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9800145e00 con 0x7f9800142390 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f98051a3640 1 --2- 172.21.15.163:0/3937427788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9800142390 0x7f98000cdd70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f97f000c400 tx=0x7f97f000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f97f67fc640 1 -- 172.21.15.163:0/3937427788 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0017070 con 0x7f9800142390 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f97f67fc640 1 -- 172.21.15.163:0/3937427788 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f0005d00 con 0x7f9800142390 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f9807c08640 1 -- 172.21.15.163:0/3937427788 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98001460e0 con 0x7f9800142390 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.772+0000 7f97f67fc640 1 -- 172.21.15.163:0/3937427788 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0007310 con 0x7f9800142390 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.773+0000 7f9807c08640 1 -- 172.21.15.163:0/3937427788 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98001466a0 con 0x7f9800142390 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.774+0000 7f9807c08640 1 -- 172.21.15.163:0/3937427788 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9800075dc0 con 0x7f9800142390 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.774+0000 7f97f67fc640 1 -- 172.21.15.163:0/3937427788 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f97f001a070 con 0x7f9800142390 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.774+0000 7f97f67fc640 1 --2- 172.21.15.163:0/3937427788 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f97e40762d0 0x7f97e4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.775+0000 7f97f67fc640 1 -- 172.21.15.163:0/3937427788 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f97f00968a0 con 0x7f9800142390 2024-05-08T06:37:36.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.775+0000 7f98059a4640 1 --2- 172.21.15.163:0/3937427788 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f97e40762d0 0x7f97e4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:36.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.775+0000 7f98059a4640 1 --2- 172.21.15.163:0/3937427788 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f97e40762d0 0x7f97e4078790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f97fc034000 tx=0x7f97fc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:36.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.779+0000 7f97f67fc640 1 -- 172.21.15.163:0/3937427788 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f97f005fb60 con 0x7f9800142390 2024-05-08T06:37:36.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.996+0000 7f9807c08640 1 -- 172.21.15.163:0/3937427788 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9800141b40 con 0x7f9800142390 2024-05-08T06:37:36.995 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.996+0000 7f97f67fc640 1 -- 172.21.15.163:0/3937427788 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5194 (secure 0 0 0) 0x7f9800141b40 con 0x7f9800142390 2024-05-08T06:37:36.995 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:36.996 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1773142301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1773142301},{"type":"v1","addr":"172.21.15.163:6835","nonce":1773142301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2952930735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2952930735},{"type":"v1","addr":"172.21.15.192:6834","nonce":2952930735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1297249355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1297249355},{"type":"v1","addr":"172.21.15.192:6835","nonce":1297249355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:37:22.365453+0000","modified":"2024-05-08T06:37:35.233488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6837/3423234723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":3423234723},{"type":"v1","addr":"172.21.15.163:6837","nonce":3423234723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-05-08T06:37:36.996 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 20 2024-05-08T06:37:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.998+0000 7f9807c08640 1 -- 172.21.15.163:0/3937427788 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f97e40762d0 msgr2=0x7f97e4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.998+0000 7f9807c08640 1 --2- 172.21.15.163:0/3937427788 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f97e40762d0 0x7f97e4078790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f97fc034000 tx=0x7f97fc02e000 comp rx=0 tx=0).stop 2024-05-08T06:37:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.998+0000 7f9807c08640 1 -- 172.21.15.163:0/3937427788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9800142390 msgr2=0x7f98000cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.998+0000 7f9807c08640 1 --2- 172.21.15.163:0/3937427788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9800142390 0x7f98000cdd70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f97f000c400 tx=0x7f97f000c800 comp rx=0 tx=0).stop 2024-05-08T06:37:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.998+0000 7f9807c08640 1 -- 172.21.15.163:0/3937427788 shutdown_connections 2024-05-08T06:37:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.998+0000 7f9807c08640 1 --2- 172.21.15.163:0/3937427788 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f97e40762d0 0x7f97e4078790 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.998+0000 7f9807c08640 1 --2- 172.21.15.163:0/3937427788 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98001636d0 0x7f9800161ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.998+0000 7f9807c08640 1 --2- 172.21.15.163:0/3937427788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9800142390 0x7f98000cdd70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.998+0000 7f9807c08640 1 --2- 172.21.15.163:0/3937427788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98000b65e0 0x7f98001f8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.998+0000 7f9807c08640 1 -- 172.21.15.163:0/3937427788 >> 172.21.15.163:0/3937427788 conn(0x7f98000b4da0 msgr2=0x7f98000722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.998+0000 7f9807c08640 1 -- 172.21.15.163:0/3937427788 shutdown_connections 2024-05-08T06:37:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:36.998+0000 7f9807c08640 1 -- 172.21.15.163:0/3937427788 wait complete. 2024-05-08T06:37:37.009 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-08T06:37:37.133 INFO:teuthology.orchestra.run.smithi163.stdout:{"message":"","return_code":0} 2024-05-08T06:37:37.134 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T06:37:37.134 DEBUG:teuthology.orchestra.run.smithi163:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event get summary 2024-05-08T06:37:37.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.177+0000 7f6c7a68f280 1 -- start start 2024-05-08T06:37:37.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.177+0000 7f6c7a68f280 1 Processor -- start 2024-05-08T06:37:37.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.177+0000 7f6c7a68f280 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec6c8ec00 0x556ec7a66000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.177+0000 7f6c7a68f280 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec6c8f800 0x556ec7a66580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.177+0000 7f6c7a68f280 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec6c8fc00 0x556ec7a66b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.177+0000 7f6c7a68f280 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556ec7a2c180 con 0x556ec6c8f800 2024-05-08T06:37:37.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7a68f280 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556ec6cf0900 con 0x556ec6c8fc00 2024-05-08T06:37:37.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7a68f280 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x556ec6cf0480 con 0x556ec6c8ec00 2024-05-08T06:37:37.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7863a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec6c8f800 0x556ec7a66580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7863a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec6c8f800 0x556ec7a66580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38874/0 (socket says 172.21.15.163:38874) 2024-05-08T06:37:37.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7863a640 1 -- 172.21.15.163:0/419862952 learned_addr learned my addr 172.21.15.163:0/419862952 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:37.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7963c640 1 --2- 172.21.15.163:0/419862952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec6c8fc00 0x556ec7a66b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c78e3b640 1 --2- 172.21.15.163:0/419862952 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec6c8ec00 0x556ec7a66000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7863a640 1 -- 172.21.15.163:0/419862952 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec6c8ec00 msgr2=0x556ec7a66000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:37.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7863a640 1 --2- 172.21.15.163:0/419862952 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec6c8ec00 0x556ec7a66000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7863a640 1 -- 172.21.15.163:0/419862952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec6c8fc00 msgr2=0x556ec7a66b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:37.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7863a640 1 --2- 172.21.15.163:0/419862952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec6c8fc00 0x556ec7a66b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7863a640 1 -- 172.21.15.163:0/419862952 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec6cc4780 con 0x556ec6c8f800 2024-05-08T06:37:37.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c78e3b640 1 --2- 172.21.15.163:0/419862952 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec6c8ec00 0x556ec7a66000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:37.185 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7863a640 1 --2- 172.21.15.163:0/419862952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec6c8f800 0x556ec7a66580 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x556ec7ac17d0 tx=0x556ec7a300c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5dca45c19e5f20a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:37.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c76636640 1 -- 172.21.15.163:0/419862952 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ec6cb4680 con 0x556ec6c8f800 2024-05-08T06:37:37.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c76636640 1 -- 172.21.15.163:0/419862952 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556ec6cba700 con 0x556ec6c8f800 2024-05-08T06:37:37.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c76636640 1 -- 172.21.15.163:0/419862952 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ec6cb4b60 con 0x556ec6c8f800 2024-05-08T06:37:37.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7a68f280 1 -- 172.21.15.163:0/419862952 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ec6cc54a0 con 0x556ec6c8f800 2024-05-08T06:37:37.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7a68f280 1 -- 172.21.15.163:0/419862952 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=0}) -- 0x556ec6cc4000 con 0x556ec6c8f800 2024-05-08T06:37:37.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c7a68f280 4 waiting for MDS map... 2024-05-08T06:37:37.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.178+0000 7f6c76636640 1 -- 172.21.15.163:0/419862952 <== mon.0 v2:172.21.15.163:3300/0 4 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x556ec7b72600 con 0x556ec6c8f800 2024-05-08T06:37:37.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.179+0000 7f6c76636640 1 -- 172.21.15.163:0/419862952 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x556ec6ce9180 con 0x556ec6c8f800 2024-05-08T06:37:37.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.179+0000 7f6c7a68f280 4 Got MDS map 20 2024-05-08T06:37:37.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.179+0000 7f6c7a68f280 10 main: JournalTool::main 2024-05-08T06:37:37.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.179+0000 7f6c7a68f280 4 main: JournalTool: connecting to RADOS... 2024-05-08T06:37:37.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.179+0000 7f6c7a68f280 1 Processor -- start 2024-05-08T06:37:37.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.179+0000 7f6c7a68f280 1 -- start start 2024-05-08T06:37:37.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c7a68f280 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec7b75400 0x556ec7a67080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c7a68f280 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec7b75800 0x556ec7a67600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c7a68f280 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec7b75c00 0x556ec7a67b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c7a68f280 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556ec6cf0180 con 0x556ec7b75800 2024-05-08T06:37:37.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c7a68f280 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556ec7a2d500 con 0x556ec7b75c00 2024-05-08T06:37:37.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c7a68f280 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x556ec7a2d380 con 0x556ec7b75400 2024-05-08T06:37:37.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c78e3b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec7b75800 0x556ec7a67600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c78e3b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec7b75800 0x556ec7a67600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38884/0 (socket says 172.21.15.163:38884) 2024-05-08T06:37:37.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c78e3b640 1 -- 172.21.15.163:0/3318384633 learned_addr learned my addr 172.21.15.163:0/3318384633 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:37.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c7963c640 1 --2- 172.21.15.163:0/3318384633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec7b75400 0x556ec7a67080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c7863a640 1 --2- 172.21.15.163:0/3318384633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec7b75c00 0x556ec7a67b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c78e3b640 1 -- 172.21.15.163:0/3318384633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec7b75400 msgr2=0x556ec7a67080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:37.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c78e3b640 1 --2- 172.21.15.163:0/3318384633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec7b75400 0x556ec7a67080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c78e3b640 1 -- 172.21.15.163:0/3318384633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec7b75c00 msgr2=0x556ec7a67b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:37.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c78e3b640 1 --2- 172.21.15.163:0/3318384633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec7b75c00 0x556ec7a67b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c78e3b640 1 -- 172.21.15.163:0/3318384633 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec6cc43c0 con 0x556ec7b75800 2024-05-08T06:37:37.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c78e3b640 1 --2- 172.21.15.163:0/3318384633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec7b75800 0x556ec7a67600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x556ec7ac18f0 tx=0x556ec7b7cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e2585cfca8decc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:37.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c74632640 1 -- 172.21.15.163:0/3318384633 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ec6cb4680 con 0x556ec7b75800 2024-05-08T06:37:37.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.180+0000 7f6c74632640 1 -- 172.21.15.163:0/3318384633 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556ec6cbafc0 con 0x556ec7b75800 2024-05-08T06:37:37.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.181+0000 7f6c74632640 1 -- 172.21.15.163:0/3318384633 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ec6cb4b60 con 0x556ec7b75800 2024-05-08T06:37:37.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.181+0000 7f6c7a68f280 1 -- 172.21.15.163:0/3318384633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec7b75800 msgr2=0x556ec7a67600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.181+0000 7f6c7a68f280 1 --2- 172.21.15.163:0/3318384633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec7b75800 0x556ec7a67600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x556ec7ac18f0 tx=0x556ec7b7cc00 comp rx=0 tx=0).stop 2024-05-08T06:37:37.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.181+0000 7f6c7a68f280 1 -- 172.21.15.163:0/3318384633 shutdown_connections 2024-05-08T06:37:37.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.181+0000 7f6c7a68f280 1 --2- 172.21.15.163:0/3318384633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec7b75c00 0x556ec7a67b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.181+0000 7f6c7a68f280 1 --2- 172.21.15.163:0/3318384633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec7b75800 0x556ec7a67600 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.181+0000 7f6c7a68f280 1 --2- 172.21.15.163:0/3318384633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec7b75400 0x556ec7a67080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.181+0000 7f6c7a68f280 1 -- 172.21.15.163:0/3318384633 >> 172.21.15.163:0/3318384633 conn(0x556ec7b75000 msgr2=0x556ec6d89000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:37.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.181+0000 7f6c7a68f280 1 -- 172.21.15.163:0/3318384633 shutdown_connections 2024-05-08T06:37:37.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.181+0000 7f6c7a68f280 1 -- 172.21.15.163:0/3318384633 wait complete. 2024-05-08T06:37:37.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.181+0000 7f6c7a68f280 1 Processor -- start 2024-05-08T06:37:37.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.181+0000 7f6c7a68f280 1 -- start start 2024-05-08T06:37:37.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7a68f280 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec7b75400 0x556ec7a67080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7a68f280 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec7b75800 0x556ec7a67600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7a68f280 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec7b75c00 0x556ec7a67b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7a68f280 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556ec7a2c780 con 0x556ec7b75400 2024-05-08T06:37:37.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7a68f280 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556ec7a2c900 con 0x556ec7b75800 2024-05-08T06:37:37.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7a68f280 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x556ec7a2c600 con 0x556ec7b75c00 2024-05-08T06:37:37.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7963c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec7b75400 0x556ec7a67080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7963c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec7b75400 0x556ec7a67080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38894/0 (socket says 172.21.15.163:38894) 2024-05-08T06:37:37.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7963c640 1 -- 172.21.15.163:0/587984140 learned_addr learned my addr 172.21.15.163:0/587984140 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:37.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c78e3b640 1 --2- 172.21.15.163:0/587984140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec7b75800 0x556ec7a67600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7863a640 1 --2- 172.21.15.163:0/587984140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec7b75c00 0x556ec7a67b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7963c640 1 -- 172.21.15.163:0/587984140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec7b75c00 msgr2=0x556ec7a67b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7963c640 1 --2- 172.21.15.163:0/587984140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec7b75c00 0x556ec7a67b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7963c640 1 -- 172.21.15.163:0/587984140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec7b75800 msgr2=0x556ec7a67600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7963c640 1 --2- 172.21.15.163:0/587984140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec7b75800 0x556ec7a67600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c7963c640 1 -- 172.21.15.163:0/587984140 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec6cc4780 con 0x556ec7b75400 2024-05-08T06:37:37.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.182+0000 7f6c78e3b640 1 --2- 172.21.15.163:0/587984140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec7b75800 0x556ec7a67600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:37.190 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7963c640 1 --2- 172.21.15.163:0/587984140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec7b75400 0x556ec7a67080 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x556ec7b77650 tx=0x556ec7a2e540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:37.191 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c72e2f640 1 -- 172.21.15.163:0/587984140 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ec6cb49c0 con 0x556ec7b75400 2024-05-08T06:37:37.191 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c72e2f640 1 -- 172.21.15.163:0/587984140 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556ec6cbaa80 con 0x556ec7b75400 2024-05-08T06:37:37.191 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 -- 172.21.15.163:0/587984140 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556ec6cc54a0 con 0x556ec7b75400 2024-05-08T06:37:37.191 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c72e2f640 1 -- 172.21.15.163:0/587984140 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ec6cb4680 con 0x556ec7b75400 2024-05-08T06:37:37.191 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 -- 172.21.15.163:0/587984140 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ec6cc4960 con 0x556ec7b75400 2024-05-08T06:37:37.191 INFO:teuthology.orchestra.run.smithi163.stderr:Error ((1) Operation not permitted) 2024-05-08T06:37:37.192 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 -1 main: Cannot run cephfs-journal-tool on an active file system! 2024-05-08T06:37:37.192 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 -- 172.21.15.163:0/587984140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec7b75400 msgr2=0x556ec7a67080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.192 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 --2- 172.21.15.163:0/587984140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec7b75400 0x556ec7a67080 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x556ec7b77650 tx=0x556ec7a2e540 comp rx=0 tx=0).stop 2024-05-08T06:37:37.192 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 -- 172.21.15.163:0/587984140 shutdown_connections 2024-05-08T06:37:37.192 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 --2- 172.21.15.163:0/587984140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec7b75c00 0x556ec7a67b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.192 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 --2- 172.21.15.163:0/587984140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec7b75800 0x556ec7a67600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.192 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 --2- 172.21.15.163:0/587984140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec7b75400 0x556ec7a67080 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 -- 172.21.15.163:0/587984140 >> 172.21.15.163:0/587984140 conn(0x556ec7b75000 msgr2=0x556ec6d89000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:37.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 -- 172.21.15.163:0/587984140 shutdown_connections 2024-05-08T06:37:37.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 -- 172.21.15.163:0/587984140 wait complete. 2024-05-08T06:37:37.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 -- 172.21.15.163:0/419862952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec6c8f800 msgr2=0x556ec7a66580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 --2- 172.21.15.163:0/419862952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec6c8f800 0x556ec7a66580 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x556ec7ac17d0 tx=0x556ec7a300c0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 -- 172.21.15.163:0/419862952 shutdown_connections 2024-05-08T06:37:37.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 --2- 172.21.15.163:0/419862952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec6c8fc00 0x556ec7a66b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.194 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 --2- 172.21.15.163:0/419862952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec6c8f800 0x556ec7a66580 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.194 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 --2- 172.21.15.163:0/419862952 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556ec6c8ec00 0x556ec7a66000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.194 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.183+0000 7f6c7a68f280 1 -- 172.21.15.163:0/419862952 >> 172.21.15.163:0/419862952 conn(0x556ec6c8f000 msgr2=0x556ec6d88800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:37.194 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.184+0000 7f6c7a68f280 1 -- 172.21.15.163:0/419862952 shutdown_connections 2024-05-08T06:37:37.194 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.184+0000 7f6c7a68f280 1 -- 172.21.15.163:0/419862952 wait complete. 2024-05-08T06:37:37.195 DEBUG:teuthology.orchestra.run:got remote process result: 255 2024-05-08T06:37:37.196 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T06:37:37.196 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T06:37:37.226 INFO:tasks.ceph.mds.a.smithi163.stderr:2024-05-08T06:37:37.227+0000 7f666b0ba640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T06:37:37.271 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T06:37:37.315 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T06:37:37.315 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T06:37:37.315 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:37:37.315 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:37.331 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18493 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T06:37:37.331 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T06:37:37.331 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:37:37.331 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:37.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056322,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497652k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12931 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497652k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T06:37:37.374 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T06:37:37.374 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:37:37.374 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:37.390 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18493 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T06:37:37.390 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T06:37:37.390 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:37:37.390 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:37.432 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056322,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497652k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12931 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497652k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T06:37:37.432 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:37.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.534+0000 7f1ffaf71640 1 Processor -- start 2024-05-08T06:37:37.533 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.534+0000 7f1ffaf71640 1 -- start start 2024-05-08T06:37:37.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.534+0000 7f1ffaf71640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ff40af310 0x7f1ff40d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.534+0000 7f1ffaf71640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ff4074830 con 0x7f1ff415d130 2024-05-08T06:37:37.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.534+0000 7f1ffaf71640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ff4073f90 con 0x7f1ff40ae180 2024-05-08T06:37:37.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.534+0000 7f1ffaf71640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff4074270 con 0x7f1ff40af310 2024-05-08T06:37:37.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff8d0d640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1ff415d130 0x7f1ff40aec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38420/0 (socket says 172.21.15.163:38420) 2024-05-08T06:37:37.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff8d0d640 1 -- 172.21.15.163:0/3968177477 learned_addr learned my addr 172.21.15.163:0/3968177477 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:37.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff3fff640 1 --2- 172.21.15.163:0/3968177477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ff40af310 0x7f1ff40d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465897358 0 0) 0x7f1ff4074830 con 0x7f1ff415d130 2024-05-08T06:37:37.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fe4003520 con 0x7f1ff415d130 2024-05-08T06:37:37.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1474931447 0 0) 0x7f1ff4073f90 con 0x7f1ff40ae180 2024-05-08T06:37:37.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ff4074830 con 0x7f1ff40ae180 2024-05-08T06:37:37.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1340743947 0 0) 0x7f1fe4003520 con 0x7f1ff415d130 2024-05-08T06:37:37.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ff40af310 msgr2=0x7f1ff40d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:37.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 --2- 172.21.15.163:0/3968177477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ff40af310 0x7f1ff40d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 >> v1:172.21.15.192:6789/0 conn(0x7f1ff40ae180 legacy=0x7f1ff406edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff4074550 con 0x7f1ff415d130 2024-05-08T06:37:37.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f1fe80016f0 con 0x7f1ff415d130 2024-05-08T06:37:37.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 >> v1:172.21.15.163:6789/0 conn(0x7f1ff415d130 legacy=0x7f1ff40aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 --2- 172.21.15.163:0/3968177477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe4003c60 0x7f1fe4004060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 --2- 172.21.15.163:0/3968177477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ff40af310 0x7f1fe4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 --2- 172.21.15.163:0/3968177477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fe4004d90 0x7f1fe4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff4074270 con 0x7f1fe4003c60 2024-05-08T06:37:37.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe400bbb0 con 0x7f1fe4004d90 2024-05-08T06:37:37.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe400bdb0 con 0x7f1ff40af310 2024-05-08T06:37:37.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff8d0d640 1 --2- 172.21.15.163:0/3968177477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe4003c60 0x7f1fe4004060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff3fff640 1 --2- 172.21.15.163:0/3968177477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ff40af310 0x7f1fe4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff950e640 1 --2- 172.21.15.163:0/3968177477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fe4004d90 0x7f1fe4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff8d0d640 1 -- 172.21.15.163:0/3968177477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ff40af310 msgr2=0x7f1fe4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:37.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff8d0d640 1 --2- 172.21.15.163:0/3968177477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ff40af310 0x7f1fe4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff8d0d640 1 -- 172.21.15.163:0/3968177477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fe4004d90 msgr2=0x7f1fe4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:37.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.535+0000 7f1ff8d0d640 1 --2- 172.21.15.163:0/3968177477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fe4004d90 0x7f1fe4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.536+0000 7f1ff8d0d640 1 -- 172.21.15.163:0/3968177477 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe4003520 con 0x7f1fe4003c60 2024-05-08T06:37:37.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.536+0000 7f1ff8d0d640 1 --2- 172.21.15.163:0/3968177477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe4003c60 0x7f1fe4004060 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1fe800c110 tx=0x7f1fe4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7dbda30a650108 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:37.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.536+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8007c40 con 0x7f1fe4003c60 2024-05-08T06:37:37.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.536+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe8005ad0 con 0x7f1fe4003c60 2024-05-08T06:37:37.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.536+0000 7f1ff37fe640 1 -- 172.21.15.163:0/3968177477 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8007070 con 0x7f1fe4003c60 2024-05-08T06:37:37.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.537+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3968177477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe4003c60 msgr2=0x7f1fe4004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.537+0000 7f1ffaf71640 1 --2- 172.21.15.163:0/3968177477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe4003c60 0x7f1fe4004060 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1fe800c110 tx=0x7f1fe4005830 comp rx=0 tx=0).stop 2024-05-08T06:37:37.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.537+0000 7f1ff950e640 1 -- 172.21.15.163:0/3968177477 reap_dead start 2024-05-08T06:37:37.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.537+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3968177477 shutdown_connections 2024-05-08T06:37:37.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.537+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3968177477 >> 172.21.15.163:0/3968177477 conn(0x7f1ff406c500 msgr2=0x7f1ff406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:37.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.537+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3968177477 shutdown_connections 2024-05-08T06:37:37.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.537+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3968177477 wait complete. 2024-05-08T06:37:37.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.538+0000 7f1ffaf71640 1 Processor -- start 2024-05-08T06:37:37.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.538+0000 7f1ffaf71640 1 -- start start 2024-05-08T06:37:37.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ffaf71640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe4003c60 0x7f1ff414d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ffaf71640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ff41545c0 0x7f1ff41f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ffaf71640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ff40cac50 0x7f1ff40cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ffaf71640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff40d08a0 con 0x7f1ff40cac50 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ffaf71640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff40d0720 con 0x7f1ff41545c0 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ffaf71640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff40d0a20 con 0x7f1fe4003c60 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ff950e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ff40cac50 0x7f1ff40cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ff950e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ff40cac50 0x7f1ff40cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38924/0 (socket says 172.21.15.163:38924) 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ff950e640 1 -- 172.21.15.163:0/3702262908 learned_addr learned my addr 172.21.15.163:0/3702262908 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ff3fff640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ff41545c0 0x7f1ff41f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ff8d0d640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe4003c60 0x7f1ff414d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ff950e640 1 -- 172.21.15.163:0/3702262908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe4003c60 msgr2=0x7f1ff414d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ff950e640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe4003c60 0x7f1ff414d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ff950e640 1 -- 172.21.15.163:0/3702262908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ff41545c0 msgr2=0x7f1ff41f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ff950e640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ff41545c0 0x7f1ff41f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ff950e640 1 -- 172.21.15.163:0/3702262908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff4155e70 con 0x7f1ff40cac50 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.539+0000 7f1ff8d0d640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe4003c60 0x7f1ff414d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.540+0000 7f1ff950e640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ff40cac50 0x7f1ff40cb0d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40b6520 tx=0x7f1fec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.540+0000 7f1ff17fa640 1 -- 172.21.15.163:0/3702262908 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec00ee70 con 0x7f1ff40cac50 2024-05-08T06:37:37.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.540+0000 7f1ff17fa640 1 -- 172.21.15.163:0/3702262908 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fec003b80 con 0x7f1ff40cac50 2024-05-08T06:37:37.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.540+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3702262908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff41f5f60 con 0x7f1ff40cac50 2024-05-08T06:37:37.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.540+0000 7f1ff17fa640 1 -- 172.21.15.163:0/3702262908 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec003d90 con 0x7f1ff40cac50 2024-05-08T06:37:37.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.540+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3702262908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff41f6520 con 0x7f1ff40cac50 2024-05-08T06:37:37.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.541+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3702262908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff4068c40 con 0x7f1ff40cac50 2024-05-08T06:37:37.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.541+0000 7f1ff17fa640 1 -- 172.21.15.163:0/3702262908 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1fec01a020 con 0x7f1ff40cac50 2024-05-08T06:37:37.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.542+0000 7f1ff17fa640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1fd80762d0 0x7f1fd8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.542+0000 7f1ff8d0d640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1fd80762d0 0x7f1fd8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.542+0000 7f1ff17fa640 1 -- 172.21.15.163:0/3702262908 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f1fec096b80 con 0x7f1ff40cac50 2024-05-08T06:37:37.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.542+0000 7f1ff8d0d640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1fd80762d0 0x7f1fd8078790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8032000 tx=0x7f1fe802d000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:37.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.546+0000 7f1ff17fa640 1 -- 172.21.15.163:0/3702262908 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1ff4068c40 con 0x7f1ff40cac50 2024-05-08T06:37:37.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.765+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3702262908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ff41f66f0 con 0x7f1ff40cac50 2024-05-08T06:37:37.765 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.766+0000 7f1ff17fa640 1 -- 172.21.15.163:0/3702262908 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5194 (secure 0 0 0) 0x7f1fec021020 con 0x7f1ff40cac50 2024-05-08T06:37:37.765 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:37.765 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1773142301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1773142301},{"type":"v1","addr":"172.21.15.163:6835","nonce":1773142301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2952930735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2952930735},{"type":"v1","addr":"172.21.15.192:6834","nonce":2952930735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1297249355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1297249355},{"type":"v1","addr":"172.21.15.192:6835","nonce":1297249355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:37:22.365453+0000","modified":"2024-05-08T06:37:35.233488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6837/3423234723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":3423234723},{"type":"v1","addr":"172.21.15.163:6837","nonce":3423234723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-05-08T06:37:37.765 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 20 2024-05-08T06:37:37.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.768+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3702262908 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1fd80762d0 msgr2=0x7f1fd8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.768+0000 7f1ffaf71640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1fd80762d0 0x7f1fd8078790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8032000 tx=0x7f1fe802d000 comp rx=0 tx=0).stop 2024-05-08T06:37:37.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.768+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3702262908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ff40cac50 msgr2=0x7f1ff40cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.768+0000 7f1ffaf71640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ff40cac50 0x7f1ff40cb0d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40b6520 tx=0x7f1fec00ec80 comp rx=0 tx=0).stop 2024-05-08T06:37:37.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.768+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3702262908 shutdown_connections 2024-05-08T06:37:37.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.768+0000 7f1ffaf71640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f1fd80762d0 0x7f1fd8078790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.768+0000 7f1ffaf71640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ff40cac50 0x7f1ff40cb0d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.768+0000 7f1ffaf71640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ff41545c0 0x7f1ff41f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.768+0000 7f1ffaf71640 1 --2- 172.21.15.163:0/3702262908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe4003c60 0x7f1ff414d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.768+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3702262908 >> 172.21.15.163:0/3702262908 conn(0x7f1ff406c500 msgr2=0x7f1ff40b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:37.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.768+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3702262908 shutdown_connections 2024-05-08T06:37:37.768 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.768+0000 7f1ffaf71640 1 -- 172.21.15.163:0/3702262908 wait complete. 2024-05-08T06:37:37.778 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:37.878 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.879+0000 7fd70bc59640 1 Processor -- start 2024-05-08T06:37:37.879 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.879+0000 7fd70bc59640 1 -- start start 2024-05-08T06:37:37.880 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.880+0000 7fd70bc59640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd704162900 0x7fd704073f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.880 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.880+0000 7fd70bc59640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd7040aef80 0x7fd7040b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.880 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.880+0000 7fd70bc59640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7040b74f0 con 0x7fd7040d2170 2024-05-08T06:37:37.881 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.880+0000 7fd70bc59640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fd704162ce0 con 0x7fd7040aef80 2024-05-08T06:37:37.881 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.880+0000 7fd70bc59640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd7040b50b0 con 0x7fd704162900 2024-05-08T06:37:37.881 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.880+0000 7fd7091f4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd7040aef80 0x7fd7040b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.881 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.880+0000 7fd7091f4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd7040aef80 0x7fd7040b6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38936/0 (socket says 172.21.15.163:38936) 2024-05-08T06:37:37.881 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.880+0000 7fd7091f4640 1 -- 172.21.15.163:0/339203515 learned_addr learned my addr 172.21.15.163:0/339203515 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:37.881 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.880+0000 7fd7099f5640 1 --2- 172.21.15.163:0/339203515 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd704162900 0x7fd704073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.881 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.881+0000 7fd7091f4640 1 -- 172.21.15.163:0/339203515 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd704162900 msgr2=0x7fd704073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:37.881 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.881+0000 7fd7091f4640 1 --2- 172.21.15.163:0/339203515 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd704162900 0x7fd704073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.881 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.881+0000 7fd7091f4640 1 -- 172.21.15.163:0/339203515 >> v1:172.21.15.192:6789/0 conn(0x7fd7040d2170 legacy=0x7fd70406c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.881 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.881+0000 7fd7091f4640 1 -- 172.21.15.163:0/339203515 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7040b5390 con 0x7fd7040aef80 2024-05-08T06:37:37.881 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.881+0000 7fd7099f5640 1 --2- 172.21.15.163:0/339203515 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd704162900 0x7fd704073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:37.881 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.881+0000 7fd7091f4640 1 --2- 172.21.15.163:0/339203515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7040aef80 0x7fd7040b6ca0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009870 tx=0x7fd6f402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81440fcbb1c40376 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:37.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.881+0000 7fd7089f3640 1 -- 172.21.15.163:0/339203515 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4037070 con 0x7fd7040aef80 2024-05-08T06:37:37.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.882+0000 7fd7089f3640 1 -- 172.21.15.163:0/339203515 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f4032440 con 0x7fd7040aef80 2024-05-08T06:37:37.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.882+0000 7fd7089f3640 1 -- 172.21.15.163:0/339203515 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4032810 con 0x7fd7040aef80 2024-05-08T06:37:37.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.882+0000 7fd70bc59640 1 -- 172.21.15.163:0/339203515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7040aef80 msgr2=0x7fd7040b6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.882+0000 7fd70bc59640 1 --2- 172.21.15.163:0/339203515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7040aef80 0x7fd7040b6ca0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009870 tx=0x7fd6f402a1c0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.882+0000 7fd70bc59640 1 -- 172.21.15.163:0/339203515 shutdown_connections 2024-05-08T06:37:37.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.882+0000 7fd70bc59640 1 --2- 172.21.15.163:0/339203515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7040aef80 0x7fd7040b6ca0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.882+0000 7fd70bc59640 1 --2- 172.21.15.163:0/339203515 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd704162900 0x7fd704073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.882+0000 7fd70bc59640 1 -- 172.21.15.163:0/339203515 >> 172.21.15.163:0/339203515 conn(0x7fd70406d610 msgr2=0x7fd70406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:37.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.882+0000 7fd70bc59640 1 -- 172.21.15.163:0/339203515 shutdown_connections 2024-05-08T06:37:37.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.882+0000 7fd70bc59640 1 -- 172.21.15.163:0/339203515 wait complete. 2024-05-08T06:37:37.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.883+0000 7fd70bc59640 1 Processor -- start 2024-05-08T06:37:37.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.883+0000 7fd70bc59640 1 -- start start 2024-05-08T06:37:37.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.884+0000 7fd70bc59640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7040aef80 0x7fd7040b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.884+0000 7fd70bc59640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7040d2170 0x7fd7041f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.884+0000 7fd70bc59640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd704162900 0x7fd7041f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.884+0000 7fd70bc59640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd7040b71e0 con 0x7fd704162900 2024-05-08T06:37:37.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.884+0000 7fd70bc59640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd70406f4d0 con 0x7fd7040aef80 2024-05-08T06:37:37.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.884+0000 7fd70bc59640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd7040d2600 con 0x7fd7040d2170 2024-05-08T06:37:37.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.884+0000 7fd70a1f6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd704162900 0x7fd7041f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.884+0000 7fd70a1f6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd704162900 0x7fd7041f52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38942/0 (socket says 172.21.15.163:38942) 2024-05-08T06:37:37.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.884+0000 7fd70a1f6640 1 -- 172.21.15.163:0/1471476340 learned_addr learned my addr 172.21.15.163:0/1471476340 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:37.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.884+0000 7fd7099f5640 1 --2- 172.21.15.163:0/1471476340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7040aef80 0x7fd7040b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.884+0000 7fd7091f4640 1 --2- 172.21.15.163:0/1471476340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7040d2170 0x7fd7041f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.885+0000 7fd70a1f6640 1 -- 172.21.15.163:0/1471476340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7040d2170 msgr2=0x7fd7041f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.885+0000 7fd70a1f6640 1 --2- 172.21.15.163:0/1471476340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7040d2170 0x7fd7041f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.885+0000 7fd70a1f6640 1 -- 172.21.15.163:0/1471476340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7040aef80 msgr2=0x7fd7040b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:37.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.885+0000 7fd70a1f6640 1 --2- 172.21.15.163:0/1471476340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7040aef80 0x7fd7040b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:37.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.885+0000 7fd70a1f6640 1 -- 172.21.15.163:0/1471476340 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7040b3b70 con 0x7fd704162900 2024-05-08T06:37:37.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.885+0000 7fd70a1f6640 1 --2- 172.21.15.163:0/1471476340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd704162900 0x7fd7041f52b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00c7a0 tx=0x7fd6fc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:37.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.885+0000 7fd6fa7fc640 1 -- 172.21.15.163:0/1471476340 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc003b70 con 0x7fd704162900 2024-05-08T06:37:37.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.885+0000 7fd6fa7fc640 1 -- 172.21.15.163:0/1471476340 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6fc003d10 con 0x7fd704162900 2024-05-08T06:37:37.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.885+0000 7fd6fa7fc640 1 -- 172.21.15.163:0/1471476340 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc00aa70 con 0x7fd704162900 2024-05-08T06:37:37.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.885+0000 7fd70bc59640 1 -- 172.21.15.163:0/1471476340 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7040b3de0 con 0x7fd704162900 2024-05-08T06:37:37.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.885+0000 7fd70bc59640 1 -- 172.21.15.163:0/1471476340 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7040b43a0 con 0x7fd704162900 2024-05-08T06:37:37.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.886+0000 7fd70bc59640 1 -- 172.21.15.163:0/1471476340 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7040b45e0 con 0x7fd704162900 2024-05-08T06:37:37.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.886+0000 7fd6fa7fc640 1 -- 172.21.15.163:0/1471476340 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd6fc00ac10 con 0x7fd704162900 2024-05-08T06:37:37.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.887+0000 7fd6fa7fc640 1 --2- 172.21.15.163:0/1471476340 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd6e80762d0 0x7fd6e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:37.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.887+0000 7fd6fa7fc640 1 -- 172.21.15.163:0/1471476340 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7fd6fc097400 con 0x7fd704162900 2024-05-08T06:37:37.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.887+0000 7fd7099f5640 1 --2- 172.21.15.163:0/1471476340 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd6e80762d0 0x7fd6e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:37.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.887+0000 7fd7099f5640 1 --2- 172.21.15.163:0/1471476340 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd6e80762d0 0x7fd6e8078790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd700006e80 tx=0x7fd700008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:37.890 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:37.892+0000 7fd6fa7fc640 1 -- 172.21.15.163:0/1471476340 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd7040b45e0 con 0x7fd704162900 2024-05-08T06:37:38.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.110+0000 7fd70bc59640 1 -- 172.21.15.163:0/1471476340 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd70406a550 con 0x7fd704162900 2024-05-08T06:37:38.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.110+0000 7fd6fa7fc640 1 -- 172.21.15.163:0/1471476340 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5194 (secure 0 0 0) 0x7fd6fc060720 con 0x7fd704162900 2024-05-08T06:37:38.110 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:38.110 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1773142301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1773142301},{"type":"v1","addr":"172.21.15.163:6835","nonce":1773142301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2952930735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2952930735},{"type":"v1","addr":"172.21.15.192:6834","nonce":2952930735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1297249355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1297249355},{"type":"v1","addr":"172.21.15.192:6835","nonce":1297249355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:37:22.365453+0000","modified":"2024-05-08T06:37:35.233488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6837/3423234723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":3423234723},{"type":"v1","addr":"172.21.15.163:6837","nonce":3423234723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-05-08T06:37:38.110 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 20 2024-05-08T06:37:38.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.112+0000 7fd70bc59640 1 -- 172.21.15.163:0/1471476340 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd6e80762d0 msgr2=0x7fd6e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.112+0000 7fd70bc59640 1 --2- 172.21.15.163:0/1471476340 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd6e80762d0 0x7fd6e8078790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd700006e80 tx=0x7fd700008040 comp rx=0 tx=0).stop 2024-05-08T06:37:38.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.112+0000 7fd70bc59640 1 -- 172.21.15.163:0/1471476340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd704162900 msgr2=0x7fd7041f52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.112+0000 7fd70bc59640 1 --2- 172.21.15.163:0/1471476340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd704162900 0x7fd7041f52b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00c7a0 tx=0x7fd6fc00cc60 comp rx=0 tx=0).stop 2024-05-08T06:37:38.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.112+0000 7fd70bc59640 1 -- 172.21.15.163:0/1471476340 shutdown_connections 2024-05-08T06:37:38.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.112+0000 7fd70bc59640 1 --2- 172.21.15.163:0/1471476340 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fd6e80762d0 0x7fd6e8078790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.112+0000 7fd70bc59640 1 --2- 172.21.15.163:0/1471476340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd704162900 0x7fd7041f52b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.112+0000 7fd70bc59640 1 --2- 172.21.15.163:0/1471476340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7040d2170 0x7fd7041f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.112+0000 7fd70bc59640 1 --2- 172.21.15.163:0/1471476340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7040aef80 0x7fd7040b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.112+0000 7fd70bc59640 1 -- 172.21.15.163:0/1471476340 >> 172.21.15.163:0/1471476340 conn(0x7fd70406d610 msgr2=0x7fd70415eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:38.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.112+0000 7fd70bc59640 1 -- 172.21.15.163:0/1471476340 shutdown_connections 2024-05-08T06:37:38.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.112+0000 7fd70bc59640 1 -- 172.21.15.163:0/1471476340 wait complete. 2024-05-08T06:37:38.123 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:37:38.222 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.224+0000 7f6a5ebc1640 1 Processor -- start 2024-05-08T06:37:38.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.224+0000 7f6a5ebc1640 1 -- start start 2024-05-08T06:37:38.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.224+0000 7f6a5ebc1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a5815f4c0 0x7f6a580d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.224+0000 7f6a5ebc1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a580b9d10 con 0x7f6a580baf20 2024-05-08T06:37:38.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.224+0000 7f6a5ebc1640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a580b9ff0 con 0x7f6a580d26f0 2024-05-08T06:37:38.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.224+0000 7f6a5ebc1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6a580d3390 con 0x7f6a5815f4c0 2024-05-08T06:37:38.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a57fff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6a580baf20 0x7f6a580d3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38436/0 (socket says 172.21.15.163:38436) 2024-05-08T06:37:38.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a57fff640 1 -- 172.21.15.163:0/403783585 learned_addr learned my addr 172.21.15.163:0/403783585 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:38.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a5c95d640 1 --2- 172.21.15.163:0/403783585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a5815f4c0 0x7f6a580d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1620591137 0 0) 0x7f6a580b9d10 con 0x7f6a580baf20 2024-05-08T06:37:38.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a40003520 con 0x7f6a580baf20 2024-05-08T06:37:38.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965083689 0 0) 0x7f6a580b9ff0 con 0x7f6a580d26f0 2024-05-08T06:37:38.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a580b9d10 con 0x7f6a580d26f0 2024-05-08T06:37:38.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1540449870 0 0) 0x7f6a40003520 con 0x7f6a580baf20 2024-05-08T06:37:38.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a5815f4c0 msgr2=0x7f6a580d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:38.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 --2- 172.21.15.163:0/403783585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a5815f4c0 0x7f6a580d21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 >> v1:172.21.15.192:6789/0 conn(0x7f6a580d26f0 legacy=0x7f6a580b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a580ba2d0 con 0x7f6a580baf20 2024-05-08T06:37:38.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f6a4c001a70 con 0x7f6a580baf20 2024-05-08T06:37:38.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 >> v1:172.21.15.163:6789/0 conn(0x7f6a580baf20 legacy=0x7f6a580d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 --2- 172.21.15.163:0/403783585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a40003c10 0x7f6a40004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 --2- 172.21.15.163:0/403783585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a40004800 0x7f6a40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 --2- 172.21.15.163:0/403783585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a5815f4c0 0x7f6a40005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6a580d3390 con 0x7f6a40003c10 2024-05-08T06:37:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4000bb50 con 0x7f6a40004800 2024-05-08T06:37:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4000bd20 con 0x7f6a5815f4c0 2024-05-08T06:37:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a5c95d640 1 --2- 172.21.15.163:0/403783585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a40003c10 0x7f6a40004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a5d15e640 1 --2- 172.21.15.163:0/403783585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a5815f4c0 0x7f6a40005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.225+0000 7f6a57fff640 1 --2- 172.21.15.163:0/403783585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a40004800 0x7f6a40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.226+0000 7f6a5c95d640 1 -- 172.21.15.163:0/403783585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a5815f4c0 msgr2=0x7f6a40005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.226+0000 7f6a5c95d640 1 --2- 172.21.15.163:0/403783585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a5815f4c0 0x7f6a40005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.226+0000 7f6a5c95d640 1 -- 172.21.15.163:0/403783585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a40004800 msgr2=0x7f6a40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.226+0000 7f6a5c95d640 1 --2- 172.21.15.163:0/403783585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a40004800 0x7f6a40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.226+0000 7f6a5c95d640 1 -- 172.21.15.163:0/403783585 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a40003520 con 0x7f6a40003c10 2024-05-08T06:37:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.226+0000 7f6a57fff640 1 --2- 172.21.15.163:0/403783585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a40004800 0x7f6a40004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.226+0000 7f6a5c95d640 1 --2- 172.21.15.163:0/403783585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a40003c10 0x7f6a40004010 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009af0 tx=0x7f6a40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30267e419dbcb81f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:38.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.226+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48036070 con 0x7f6a40003c10 2024-05-08T06:37:38.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.226+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a48031400 con 0x7f6a40003c10 2024-05-08T06:37:38.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.226+0000 7f6a577fe640 1 -- 172.21.15.163:0/403783585 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48009690 con 0x7f6a40003c10 2024-05-08T06:37:38.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.227+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/403783585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a40003c10 msgr2=0x7f6a40004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.227+0000 7f6a5ebc1640 1 --2- 172.21.15.163:0/403783585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a40003c10 0x7f6a40004010 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009af0 tx=0x7f6a40005830 comp rx=0 tx=0).stop 2024-05-08T06:37:38.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.227+0000 7f6a5d15e640 1 -- 172.21.15.163:0/403783585 reap_dead start 2024-05-08T06:37:38.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.227+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/403783585 shutdown_connections 2024-05-08T06:37:38.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.227+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/403783585 >> 172.21.15.163:0/403783585 conn(0x7f6a58073e70 msgr2=0x7f6a58074280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:38.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.227+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/403783585 shutdown_connections 2024-05-08T06:37:38.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.227+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/403783585 wait complete. 2024-05-08T06:37:38.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.228+0000 7f6a5ebc1640 1 Processor -- start 2024-05-08T06:37:38.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.228+0000 7f6a5ebc1640 1 -- start start 2024-05-08T06:37:38.233 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.229+0000 7f6a5ebc1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a40003c10 0x7f6a580c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.233 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.229+0000 7f6a5ebc1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a58154520 0x7f6a58153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.233 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.229+0000 7f6a5ebc1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a580ca560 0x7f6a58161460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.233 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.229+0000 7f6a5ebc1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6a580b9ac0 con 0x7f6a58154520 2024-05-08T06:37:38.233 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.229+0000 7f6a5ebc1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6a580b9940 con 0x7f6a580ca560 2024-05-08T06:37:38.233 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.229+0000 7f6a5ebc1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6a580d21b0 con 0x7f6a40003c10 2024-05-08T06:37:38.233 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.229+0000 7f6a57fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a58154520 0x7f6a58153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.233 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.229+0000 7f6a57fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a58154520 0x7f6a58153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38958/0 (socket says 172.21.15.163:38958) 2024-05-08T06:37:38.233 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.229+0000 7f6a57fff640 1 -- 172.21.15.163:0/38915410 learned_addr learned my addr 172.21.15.163:0/38915410 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:38.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.229+0000 7f6a5c95d640 1 --2- 172.21.15.163:0/38915410 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a40003c10 0x7f6a580c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.229+0000 7f6a5d15e640 1 --2- 172.21.15.163:0/38915410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a580ca560 0x7f6a58161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.230+0000 7f6a57fff640 1 -- 172.21.15.163:0/38915410 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a40003c10 msgr2=0x7f6a580c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.230+0000 7f6a57fff640 1 --2- 172.21.15.163:0/38915410 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a40003c10 0x7f6a580c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.230+0000 7f6a57fff640 1 -- 172.21.15.163:0/38915410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a580ca560 msgr2=0x7f6a58161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.230+0000 7f6a57fff640 1 --2- 172.21.15.163:0/38915410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a580ca560 0x7f6a58161460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.230+0000 7f6a57fff640 1 -- 172.21.15.163:0/38915410 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a580c94c0 con 0x7f6a58154520 2024-05-08T06:37:38.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.230+0000 7f6a57fff640 1 --2- 172.21.15.163:0/38915410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a58154520 0x7f6a58153c10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c007a30 tx=0x7f6a4c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:38.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.230+0000 7f6a557fa640 1 -- 172.21.15.163:0/38915410 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00ee50 con 0x7f6a58154520 2024-05-08T06:37:38.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.230+0000 7f6a557fa640 1 -- 172.21.15.163:0/38915410 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a4c018070 con 0x7f6a58154520 2024-05-08T06:37:38.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.230+0000 7f6a557fa640 1 -- 172.21.15.163:0/38915410 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c013640 con 0x7f6a58154520 2024-05-08T06:37:38.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.230+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/38915410 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a580c97a0 con 0x7f6a58154520 2024-05-08T06:37:38.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.230+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/38915410 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a580c9d60 con 0x7f6a58154520 2024-05-08T06:37:38.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.231+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/38915410 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a5806a550 con 0x7f6a58154520 2024-05-08T06:37:38.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.231+0000 7f6a557fa640 1 -- 172.21.15.163:0/38915410 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6a4c0137e0 con 0x7f6a58154520 2024-05-08T06:37:38.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.232+0000 7f6a557fa640 1 --2- 172.21.15.163:0/38915410 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6a440762d0 0x7f6a44078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.232+0000 7f6a557fa640 1 -- 172.21.15.163:0/38915410 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f6a4c097c10 con 0x7f6a58154520 2024-05-08T06:37:38.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.232+0000 7f6a5c95d640 1 --2- 172.21.15.163:0/38915410 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6a440762d0 0x7f6a44078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.232+0000 7f6a5c95d640 1 --2- 172.21.15.163:0/38915410 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6a440762d0 0x7f6a44078790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6a48033000 tx=0x7f6a48008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:38.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.236+0000 7f6a557fa640 1 -- 172.21.15.163:0/38915410 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6a4c060e30 con 0x7f6a58154520 2024-05-08T06:37:38.419 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.420+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/38915410 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a5815bdf0 con 0x7f6a58154520 2024-05-08T06:37:38.420 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.421+0000 7f6a557fa640 1 -- 172.21.15.163:0/38915410 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16822 (secure 0 0 0) 0x7f6a4c065130 con 0x7f6a58154520 2024-05-08T06:37:38.420 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:38.420 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":32,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:37:31.216129+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:37:20.131664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T06:37:21.318802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/880835577":"2024-05-09T06:37:31.216100+0000","172.21.15.163:6836/880835577":"2024-05-09T06:37:31.216100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:37:38.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.423+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/38915410 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6a440762d0 msgr2=0x7f6a44078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.423+0000 7f6a5ebc1640 1 --2- 172.21.15.163:0/38915410 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6a440762d0 0x7f6a44078790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6a48033000 tx=0x7f6a48008030 comp rx=0 tx=0).stop 2024-05-08T06:37:38.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.423+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/38915410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a58154520 msgr2=0x7f6a58153c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.423+0000 7f6a5ebc1640 1 --2- 172.21.15.163:0/38915410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a58154520 0x7f6a58153c10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c007a30 tx=0x7f6a4c00e530 comp rx=0 tx=0).stop 2024-05-08T06:37:38.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.423+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/38915410 shutdown_connections 2024-05-08T06:37:38.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.423+0000 7f6a5ebc1640 1 --2- 172.21.15.163:0/38915410 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6a440762d0 0x7f6a44078790 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.423+0000 7f6a5ebc1640 1 --2- 172.21.15.163:0/38915410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a580ca560 0x7f6a58161460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.423+0000 7f6a5ebc1640 1 --2- 172.21.15.163:0/38915410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a58154520 0x7f6a58153c10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.423+0000 7f6a5ebc1640 1 --2- 172.21.15.163:0/38915410 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a40003c10 0x7f6a580c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.423+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/38915410 >> 172.21.15.163:0/38915410 conn(0x7f6a58073e70 msgr2=0x7f6a580d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:38.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.423+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/38915410 shutdown_connections 2024-05-08T06:37:38.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.423+0000 7f6a5ebc1640 1 -- 172.21.15.163:0/38915410 wait complete. 2024-05-08T06:37:38.434 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T06:37:38.434 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:38.534 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.535+0000 7f42fed12640 1 Processor -- start 2024-05-08T06:37:38.534 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.535+0000 7f42fed12640 1 -- start start 2024-05-08T06:37:38.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.536+0000 7f42fed12640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f42f815c020 0x7f42f8073720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.536+0000 7f42fed12640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42f80b48b0 0x7f42f80afc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.536+0000 7f42fed12640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42f8072f80 con 0x7f42f80ae180 2024-05-08T06:37:38.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.536+0000 7f42fed12640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f42f80ae700 con 0x7f42f80b48b0 2024-05-08T06:37:38.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.536+0000 7f42fed12640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f42f8073290 con 0x7f42f815c020 2024-05-08T06:37:38.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.536+0000 7f42fd2af640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42f80b48b0 0x7f42f80afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.536+0000 7f42fd2af640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42f80b48b0 0x7f42f80afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38968/0 (socket says 172.21.15.163:38968) 2024-05-08T06:37:38.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.536+0000 7f42fd2af640 1 -- 172.21.15.163:0/2887880604 learned_addr learned my addr 172.21.15.163:0/2887880604 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:38.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.536+0000 7f42fd2af640 1 -- 172.21.15.163:0/2887880604 >> v2:172.21.15.192:3300/0 conn(0x7f42f815c020 msgr2=0x7f42f8073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:38.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.536+0000 7f42fd2af640 1 --2- 172.21.15.163:0/2887880604 >> v2:172.21.15.192:3300/0 conn(0x7f42f815c020 0x7f42f8073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.536+0000 7f42fd2af640 1 -- 172.21.15.163:0/2887880604 >> v1:172.21.15.192:6789/0 conn(0x7f42f80ae180 legacy=0x7f42f80b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:38.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.536+0000 7f42fd2af640 1 -- 172.21.15.163:0/2887880604 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f8074430 con 0x7f42f80b48b0 2024-05-08T06:37:38.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.537+0000 7f42fd2af640 1 --2- 172.21.15.163:0/2887880604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f80b48b0 0x7f42f80afc00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f42f80b3f10 tx=0x7f42f002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a53a7764af1868 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:38.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.537+0000 7f42f77fe640 1 -- 172.21.15.163:0/2887880604 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0036070 con 0x7f42f80b48b0 2024-05-08T06:37:38.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.537+0000 7f42f77fe640 1 -- 172.21.15.163:0/2887880604 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f0031600 con 0x7f42f80b48b0 2024-05-08T06:37:38.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.537+0000 7f42f77fe640 1 -- 172.21.15.163:0/2887880604 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f00319d0 con 0x7f42f80b48b0 2024-05-08T06:37:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.538+0000 7f42fed12640 1 -- 172.21.15.163:0/2887880604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f80b48b0 msgr2=0x7f42f80afc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.538+0000 7f42fed12640 1 --2- 172.21.15.163:0/2887880604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f80b48b0 0x7f42f80afc00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f42f80b3f10 tx=0x7f42f002a570 comp rx=0 tx=0).stop 2024-05-08T06:37:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.538+0000 7f42fed12640 1 -- 172.21.15.163:0/2887880604 shutdown_connections 2024-05-08T06:37:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.538+0000 7f42fed12640 1 --2- 172.21.15.163:0/2887880604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f80b48b0 0x7f42f80afc00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.538+0000 7f42fed12640 1 --2- 172.21.15.163:0/2887880604 >> v2:172.21.15.192:3300/0 conn(0x7f42f815c020 0x7f42f8073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.538+0000 7f42fed12640 1 -- 172.21.15.163:0/2887880604 >> 172.21.15.163:0/2887880604 conn(0x7f42f806c500 msgr2=0x7f42f806c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.538+0000 7f42fed12640 1 -- 172.21.15.163:0/2887880604 shutdown_connections 2024-05-08T06:37:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.538+0000 7f42fed12640 1 -- 172.21.15.163:0/2887880604 wait complete. 2024-05-08T06:37:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.539+0000 7f42fed12640 1 Processor -- start 2024-05-08T06:37:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.539+0000 7f42fed12640 1 -- start start 2024-05-08T06:37:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fed12640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f80ae180 0x7f42f8153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fed12640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42f80b48b0 0x7f42f8149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fed12640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42f815c020 0x7f42f814f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fed12640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f42f80d0860 con 0x7f42f80ae180 2024-05-08T06:37:38.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fed12640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f42f815c400 con 0x7f42f815c020 2024-05-08T06:37:38.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fed12640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f42f8073c60 con 0x7f42f80b48b0 2024-05-08T06:37:38.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fcaae640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f80ae180 0x7f42f8153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fcaae640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f80ae180 0x7f42f8153720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38970/0 (socket says 172.21.15.163:38970) 2024-05-08T06:37:38.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fcaae640 1 -- 172.21.15.163:0/4154411532 learned_addr learned my addr 172.21.15.163:0/4154411532 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:38.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fd2af640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42f815c020 0x7f42f814f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42f7fff640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42f80b48b0 0x7f42f8149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fcaae640 1 -- 172.21.15.163:0/4154411532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42f80b48b0 msgr2=0x7f42f8149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fcaae640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42f80b48b0 0x7f42f8149f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fcaae640 1 -- 172.21.15.163:0/4154411532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42f815c020 msgr2=0x7f42f814f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fcaae640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42f815c020 0x7f42f814f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fcaae640 1 -- 172.21.15.163:0/4154411532 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f8151180 con 0x7f42f80ae180 2024-05-08T06:37:38.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42f7fff640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42f80b48b0 0x7f42f8149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:38.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fcaae640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f80ae180 0x7f42f8153720 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00b410 tx=0x7f42ec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:38.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42f57fa640 1 -- 172.21.15.163:0/4154411532 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec015020 con 0x7f42f80ae180 2024-05-08T06:37:38.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42f57fa640 1 -- 172.21.15.163:0/4154411532 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42ec011850 con 0x7f42f80ae180 2024-05-08T06:37:38.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42f57fa640 1 -- 172.21.15.163:0/4154411532 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec011c40 con 0x7f42f80ae180 2024-05-08T06:37:38.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fed12640 1 -- 172.21.15.163:0/4154411532 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f8151460 con 0x7f42f80ae180 2024-05-08T06:37:38.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.540+0000 7f42fed12640 1 -- 172.21.15.163:0/4154411532 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f8151a20 con 0x7f42f80ae180 2024-05-08T06:37:38.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.541+0000 7f42fed12640 1 -- 172.21.15.163:0/4154411532 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f806a550 con 0x7f42f80ae180 2024-05-08T06:37:38.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.542+0000 7f42f57fa640 1 -- 172.21.15.163:0/4154411532 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f42ec018020 con 0x7f42f80ae180 2024-05-08T06:37:38.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.542+0000 7f42f57fa640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f42dc0762d0 0x7f42dc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.543+0000 7f42f57fa640 1 -- 172.21.15.163:0/4154411532 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f42ec097490 con 0x7f42f80ae180 2024-05-08T06:37:38.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.543+0000 7f42f7fff640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f42dc0762d0 0x7f42dc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.543+0000 7f42f7fff640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f42dc0762d0 0x7f42dc078790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f42e4006fd0 tx=0x7f42e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:38.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.547+0000 7f42f57fa640 1 -- 172.21.15.163:0/4154411532 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f42ec01f020 con 0x7f42f80ae180 2024-05-08T06:37:38.766 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.767+0000 7f42fed12640 1 -- 172.21.15.163:0/4154411532 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42f815c580 con 0x7f42f80ae180 2024-05-08T06:37:38.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.767+0000 7f42f57fa640 1 -- 172.21.15.163:0/4154411532 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5194 (secure 0 0 0) 0x7f42ec060e20 con 0x7f42f80ae180 2024-05-08T06:37:38.767 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:38.767 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1773142301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1773142301},{"type":"v1","addr":"172.21.15.163:6835","nonce":1773142301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2952930735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2952930735},{"type":"v1","addr":"172.21.15.192:6834","nonce":2952930735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1297249355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1297249355},{"type":"v1","addr":"172.21.15.192:6835","nonce":1297249355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:37:22.365453+0000","modified":"2024-05-08T06:37:35.233488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6837/3423234723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":3423234723},{"type":"v1","addr":"172.21.15.163:6837","nonce":3423234723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-05-08T06:37:38.767 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 20 2024-05-08T06:37:38.769 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.770+0000 7f42fed12640 1 -- 172.21.15.163:0/4154411532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f42dc0762d0 msgr2=0x7f42dc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.769 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.770+0000 7f42fed12640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f42dc0762d0 0x7f42dc078790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f42e4006fd0 tx=0x7f42e4008040 comp rx=0 tx=0).stop 2024-05-08T06:37:38.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.770+0000 7f42fed12640 1 -- 172.21.15.163:0/4154411532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f80ae180 msgr2=0x7f42f8153720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.770+0000 7f42fed12640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f80ae180 0x7f42f8153720 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00b410 tx=0x7f42ec00b8d0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.770+0000 7f42fed12640 1 -- 172.21.15.163:0/4154411532 shutdown_connections 2024-05-08T06:37:38.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.770+0000 7f42fed12640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f42dc0762d0 0x7f42dc078790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.770+0000 7f42fed12640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42f815c020 0x7f42f814f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.770+0000 7f42fed12640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42f80b48b0 0x7f42f8149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.770+0000 7f42fed12640 1 --2- 172.21.15.163:0/4154411532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f80ae180 0x7f42f8153720 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.770+0000 7f42fed12640 1 -- 172.21.15.163:0/4154411532 >> 172.21.15.163:0/4154411532 conn(0x7f42f806c500 msgr2=0x7f42f815cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:38.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.770+0000 7f42fed12640 1 -- 172.21.15.163:0/4154411532 shutdown_connections 2024-05-08T06:37:38.770 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.770+0000 7f42fed12640 1 -- 172.21.15.163:0/4154411532 wait complete. 2024-05-08T06:37:38.780 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:37:38.880 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.881+0000 7ff783714640 1 Processor -- start 2024-05-08T06:37:38.881 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.882+0000 7ff783714640 1 -- start start 2024-05-08T06:37:38.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.882+0000 7ff783714640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff77c0d0a00 0x7ff77c0b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.882+0000 7ff783714640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff77c0cfc10 0x7ff77c0b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.882+0000 7ff783714640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff77c0d0080 con 0x7ff77c15f760 2024-05-08T06:37:38.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.882+0000 7ff783714640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7ff77c15fb40 con 0x7ff77c0d0a00 2024-05-08T06:37:38.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.883+0000 7ff783714640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff77c0d36f0 con 0x7ff77c0cfc10 2024-05-08T06:37:38.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.883+0000 7ff780caf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff77c0d0a00 0x7ff77c0b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.883+0000 7ff7814b0640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff77c15f760 0x7ff77c0d02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38442/0 (socket says 172.21.15.163:38442) 2024-05-08T06:37:38.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.883+0000 7ff780caf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff77c0d0a00 0x7ff77c0b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38974/0 (socket says 172.21.15.163:38974) 2024-05-08T06:37:38.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.883+0000 7ff7814b0640 1 -- 172.21.15.163:0/507854594 learned_addr learned my addr 172.21.15.163:0/507854594 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:38.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.883+0000 7ff781cb1640 1 --2- 172.21.15.163:0/507854594 >> v2:172.21.15.192:3300/0 conn(0x7ff77c0cfc10 0x7ff77c0b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.883+0000 7ff780caf640 1 -- 172.21.15.163:0/507854594 >> v2:172.21.15.192:3300/0 conn(0x7ff77c0cfc10 msgr2=0x7ff77c0b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:38.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.883+0000 7ff780caf640 1 --2- 172.21.15.163:0/507854594 >> v2:172.21.15.192:3300/0 conn(0x7ff77c0cfc10 0x7ff77c0b8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.883+0000 7ff780caf640 1 -- 172.21.15.163:0/507854594 >> v1:172.21.15.163:6789/0 conn(0x7ff77c15f760 legacy=0x7ff77c0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.883+0000 7ff773fff640 1 -- 172.21.15.163:0/507854594 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2019702747 0 0) 0x7ff77c0d0080 con 0x7ff77c15f760 2024-05-08T06:37:38.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.883+0000 7ff780caf640 1 -- 172.21.15.163:0/507854594 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c0d39d0 con 0x7ff77c0d0a00 2024-05-08T06:37:38.884 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.883+0000 7ff781cb1640 1 --2- 172.21.15.163:0/507854594 >> v2:172.21.15.192:3300/0 conn(0x7ff77c0cfc10 0x7ff77c0b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:38.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.884+0000 7ff780caf640 1 --2- 172.21.15.163:0/507854594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff77c0d0a00 0x7ff77c0b4770 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0b8170 tx=0x7ff76c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14edd17fbf2ea2f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:38.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.884+0000 7ff773fff640 1 -- 172.21.15.163:0/507854594 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c037070 con 0x7ff77c0d0a00 2024-05-08T06:37:38.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.884+0000 7ff773fff640 1 -- 172.21.15.163:0/507854594 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76c032440 con 0x7ff77c0d0a00 2024-05-08T06:37:38.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.884+0000 7ff773fff640 1 -- 172.21.15.163:0/507854594 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c02ae60 con 0x7ff77c0d0a00 2024-05-08T06:37:38.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.884+0000 7ff783714640 1 -- 172.21.15.163:0/507854594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff77c0d0a00 msgr2=0x7ff77c0b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.887 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.884+0000 7ff783714640 1 --2- 172.21.15.163:0/507854594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff77c0d0a00 0x7ff77c0b4770 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0b8170 tx=0x7ff76c02a110 comp rx=0 tx=0).stop 2024-05-08T06:37:38.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.884+0000 7ff783714640 1 -- 172.21.15.163:0/507854594 shutdown_connections 2024-05-08T06:37:38.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.884+0000 7ff783714640 1 --2- 172.21.15.163:0/507854594 >> v2:172.21.15.192:3300/0 conn(0x7ff77c0cfc10 0x7ff77c0b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.884+0000 7ff783714640 1 --2- 172.21.15.163:0/507854594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff77c0d0a00 0x7ff77c0b4770 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.884+0000 7ff783714640 1 -- 172.21.15.163:0/507854594 >> 172.21.15.163:0/507854594 conn(0x7ff77c0740e0 msgr2=0x7ff77c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:38.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.885+0000 7ff783714640 1 -- 172.21.15.163:0/507854594 shutdown_connections 2024-05-08T06:37:38.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.885+0000 7ff783714640 1 -- 172.21.15.163:0/507854594 wait complete. 2024-05-08T06:37:38.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.886+0000 7ff783714640 1 Processor -- start 2024-05-08T06:37:38.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.886+0000 7ff783714640 1 -- start start 2024-05-08T06:37:38.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.886+0000 7ff783714640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff77c0cfc10 0x7ff77c151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.886+0000 7ff783714640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff77c0d0a00 0x7ff77c159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.886+0000 7ff783714640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff77c15f760 0x7ff77c0b9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.886+0000 7ff783714640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c0d4270 con 0x7ff77c0d0a00 2024-05-08T06:37:38.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.886+0000 7ff783714640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c0d3f90 con 0x7ff77c15f760 2024-05-08T06:37:38.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.886+0000 7ff783714640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff77c0d4790 con 0x7ff77c0cfc10 2024-05-08T06:37:38.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.886+0000 7ff780caf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff77c0d0a00 0x7ff77c159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff780caf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff77c0d0a00 0x7ff77c159130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38980/0 (socket says 172.21.15.163:38980) 2024-05-08T06:37:38.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff780caf640 1 -- 172.21.15.163:0/3423015611 learned_addr learned my addr 172.21.15.163:0/3423015611 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:38.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff781cb1640 1 --2- 172.21.15.163:0/3423015611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff77c15f760 0x7ff77c0b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff7814b0640 1 --2- 172.21.15.163:0/3423015611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff77c0cfc10 0x7ff77c151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff780caf640 1 -- 172.21.15.163:0/3423015611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff77c0cfc10 msgr2=0x7ff77c151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff780caf640 1 --2- 172.21.15.163:0/3423015611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff77c0cfc10 0x7ff77c151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff780caf640 1 -- 172.21.15.163:0/3423015611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff77c15f760 msgr2=0x7ff77c0b9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:38.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff780caf640 1 --2- 172.21.15.163:0/3423015611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff77c15f760 0x7ff77c0b9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:38.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff780caf640 1 -- 172.21.15.163:0/3423015611 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c151d60 con 0x7ff77c0d0a00 2024-05-08T06:37:38.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff780caf640 1 --2- 172.21.15.163:0/3423015611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff77c0d0a00 0x7ff77c159130 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff76c000c00 tx=0x7ff76c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:38.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff771ffb640 1 -- 172.21.15.163:0/3423015611 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c037040 con 0x7ff77c0d0a00 2024-05-08T06:37:38.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff771ffb640 1 -- 172.21.15.163:0/3423015611 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76c042070 con 0x7ff77c0d0a00 2024-05-08T06:37:38.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff771ffb640 1 -- 172.21.15.163:0/3423015611 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c030a50 con 0x7ff77c0d0a00 2024-05-08T06:37:38.890 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff783714640 1 -- 172.21.15.163:0/3423015611 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77c1599a0 con 0x7ff77c0d0a00 2024-05-08T06:37:38.890 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.887+0000 7ff783714640 1 -- 172.21.15.163:0/3423015611 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77c159f60 con 0x7ff77c0d0a00 2024-05-08T06:37:38.890 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.888+0000 7ff783714640 1 -- 172.21.15.163:0/3423015611 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77c06a550 con 0x7ff77c0d0a00 2024-05-08T06:37:38.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.889+0000 7ff771ffb640 1 -- 172.21.15.163:0/3423015611 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff76c030bf0 con 0x7ff77c0d0a00 2024-05-08T06:37:38.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.889+0000 7ff771ffb640 1 --2- 172.21.15.163:0/3423015611 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff7600762d0 0x7ff760078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:38.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.889+0000 7ff771ffb640 1 -- 172.21.15.163:0/3423015611 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7ff76c0b8050 con 0x7ff77c0d0a00 2024-05-08T06:37:38.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.889+0000 7ff7814b0640 1 --2- 172.21.15.163:0/3423015611 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff7600762d0 0x7ff760078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:38.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.890+0000 7ff7814b0640 1 --2- 172.21.15.163:0/3423015611 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff7600762d0 0x7ff760078790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff778000f60 tx=0x7ff778007b10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:38.892 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:38.894+0000 7ff771ffb640 1 -- 172.21.15.163:0/3423015611 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff76c0819e0 con 0x7ff77c0d0a00 2024-05-08T06:37:39.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.113+0000 7ff783714640 1 -- 172.21.15.163:0/3423015611 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff77c0c8a80 con 0x7ff77c0d0a00 2024-05-08T06:37:39.112 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.113+0000 7ff771ffb640 1 -- 172.21.15.163:0/3423015611 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5194 (secure 0 0 0) 0x7ff76c0812e0 con 0x7ff77c0d0a00 2024-05-08T06:37:39.113 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:39.113 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1773142301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1773142301},{"type":"v1","addr":"172.21.15.163:6835","nonce":1773142301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2952930735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2952930735},{"type":"v1","addr":"172.21.15.192:6834","nonce":2952930735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1297249355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1297249355},{"type":"v1","addr":"172.21.15.192:6835","nonce":1297249355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T06:37:22.365453+0000","modified":"2024-05-08T06:37:35.233488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6837/3423234723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":3423234723},{"type":"v1","addr":"172.21.15.163:6837","nonce":3423234723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-05-08T06:37:39.113 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 20 2024-05-08T06:37:39.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.116+0000 7ff783714640 1 -- 172.21.15.163:0/3423015611 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff7600762d0 msgr2=0x7ff760078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.116+0000 7ff783714640 1 --2- 172.21.15.163:0/3423015611 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff7600762d0 0x7ff760078790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff778000f60 tx=0x7ff778007b10 comp rx=0 tx=0).stop 2024-05-08T06:37:39.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.116+0000 7ff783714640 1 -- 172.21.15.163:0/3423015611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff77c0d0a00 msgr2=0x7ff77c159130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.116+0000 7ff783714640 1 --2- 172.21.15.163:0/3423015611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff77c0d0a00 0x7ff77c159130 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff76c000c00 tx=0x7ff76c0046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.116+0000 7ff783714640 1 -- 172.21.15.163:0/3423015611 shutdown_connections 2024-05-08T06:37:39.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.116+0000 7ff783714640 1 --2- 172.21.15.163:0/3423015611 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff7600762d0 0x7ff760078790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.116+0000 7ff783714640 1 --2- 172.21.15.163:0/3423015611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff77c15f760 0x7ff77c0b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.116+0000 7ff783714640 1 --2- 172.21.15.163:0/3423015611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff77c0d0a00 0x7ff77c159130 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.116+0000 7ff783714640 1 --2- 172.21.15.163:0/3423015611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff77c0cfc10 0x7ff77c151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.116+0000 7ff783714640 1 -- 172.21.15.163:0/3423015611 >> 172.21.15.163:0/3423015611 conn(0x7ff77c0740e0 msgr2=0x7ff77c0cc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:39.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.116+0000 7ff783714640 1 -- 172.21.15.163:0/3423015611 shutdown_connections 2024-05-08T06:37:39.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.116+0000 7ff783714640 1 -- 172.21.15.163:0/3423015611 wait complete. 2024-05-08T06:37:39.126 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:37:39.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.227+0000 7faad0672640 1 Processor -- start 2024-05-08T06:37:39.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.228+0000 7faad0672640 1 -- start start 2024-05-08T06:37:39.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.229+0000 7faad0672640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faac806da40 0x7faac806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.229+0000 7faad0672640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faac806ef80 0x7faac815e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.229+0000 7faad0672640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faac806f410 con 0x7faac815d600 2024-05-08T06:37:39.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.229+0000 7faad0672640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7faac806f6c0 con 0x7faac806da40 2024-05-08T06:37:39.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.229+0000 7faad0672640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7faac815de20 con 0x7faac806ef80 2024-05-08T06:37:39.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.229+0000 7faacdc0d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faac806da40 0x7faac806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.229+0000 7faacdc0d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faac806da40 0x7faac806ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38996/0 (socket says 172.21.15.163:38996) 2024-05-08T06:37:39.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.229+0000 7faacdc0d640 1 -- 172.21.15.163:0/43912541 learned_addr learned my addr 172.21.15.163:0/43912541 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:39.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.229+0000 7faacec0f640 1 --2- 172.21.15.163:0/43912541 >> v2:172.21.15.192:3300/0 conn(0x7faac806ef80 0x7faac815e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.229+0000 7faacdc0d640 1 -- 172.21.15.163:0/43912541 >> v2:172.21.15.192:3300/0 conn(0x7faac806ef80 msgr2=0x7faac815e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:39.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.229+0000 7faacdc0d640 1 --2- 172.21.15.163:0/43912541 >> v2:172.21.15.192:3300/0 conn(0x7faac806ef80 0x7faac815e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.229+0000 7faacdc0d640 1 -- 172.21.15.163:0/43912541 >> v1:172.21.15.192:6789/0 conn(0x7faac815d600 legacy=0x7faac806d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.229+0000 7faacdc0d640 1 -- 172.21.15.163:0/43912541 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac815e100 con 0x7faac806da40 2024-05-08T06:37:39.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.230+0000 7faacdc0d640 1 --2- 172.21.15.163:0/43912541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac806da40 0x7faac806ea40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faac80d0b60 tx=0x7faab802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151b39768846970e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:39.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.230+0000 7faacd40c640 1 -- 172.21.15.163:0/43912541 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8037070 con 0x7faac806da40 2024-05-08T06:37:39.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.230+0000 7faacd40c640 1 -- 172.21.15.163:0/43912541 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab8032440 con 0x7faac806da40 2024-05-08T06:37:39.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.230+0000 7faacd40c640 1 -- 172.21.15.163:0/43912541 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8032810 con 0x7faac806da40 2024-05-08T06:37:39.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.231+0000 7faad0672640 1 -- 172.21.15.163:0/43912541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac806da40 msgr2=0x7faac806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.231+0000 7faad0672640 1 --2- 172.21.15.163:0/43912541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac806da40 0x7faac806ea40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faac80d0b60 tx=0x7faab802a270 comp rx=0 tx=0).stop 2024-05-08T06:37:39.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.231+0000 7faad0672640 1 -- 172.21.15.163:0/43912541 shutdown_connections 2024-05-08T06:37:39.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.231+0000 7faad0672640 1 --2- 172.21.15.163:0/43912541 >> v2:172.21.15.192:3300/0 conn(0x7faac806ef80 0x7faac815e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.231+0000 7faad0672640 1 --2- 172.21.15.163:0/43912541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac806da40 0x7faac806ea40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.231+0000 7faad0672640 1 -- 172.21.15.163:0/43912541 >> 172.21.15.163:0/43912541 conn(0x7faac8072ba0 msgr2=0x7faac8072f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:39.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.231+0000 7faad0672640 1 -- 172.21.15.163:0/43912541 shutdown_connections 2024-05-08T06:37:39.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.231+0000 7faad0672640 1 -- 172.21.15.163:0/43912541 wait complete. 2024-05-08T06:37:39.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.232+0000 7faad0672640 1 Processor -- start 2024-05-08T06:37:39.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.232+0000 7faad0672640 1 -- start start 2024-05-08T06:37:39.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faad0672640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faac806da40 0x7faac8153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faad0672640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faac806ef80 0x7faac8152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faad0672640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac815d600 0x7faac815fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faad0672640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7faac815d9e0 con 0x7faac815d600 2024-05-08T06:37:39.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faad0672640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faac806f9a0 con 0x7faac806da40 2024-05-08T06:37:39.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faad0672640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7faac815db60 con 0x7faac806ef80 2024-05-08T06:37:39.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faacec0f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac815d600 0x7faac815fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faacec0f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac815d600 0x7faac815fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39000/0 (socket says 172.21.15.163:39000) 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faacec0f640 1 -- 172.21.15.163:0/2436651751 learned_addr learned my addr 172.21.15.163:0/2436651751 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faace40e640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faac806da40 0x7faac8153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faacdc0d640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faac806ef80 0x7faac8152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faacec0f640 1 -- 172.21.15.163:0/2436651751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faac806ef80 msgr2=0x7faac8152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faacec0f640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faac806ef80 0x7faac8152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faacec0f640 1 -- 172.21.15.163:0/2436651751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faac806da40 msgr2=0x7faac8153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faacec0f640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faac806da40 0x7faac8153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faacec0f640 1 -- 172.21.15.163:0/2436651751 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac81e7840 con 0x7faac815d600 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faace40e640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faac806da40 0x7faac8153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faacec0f640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac815d600 0x7faac815fd60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faac000efc0 tx=0x7faac000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faabeffd640 1 -- 172.21.15.163:0/2436651751 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0018020 con 0x7faac815d600 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faabeffd640 1 -- 172.21.15.163:0/2436651751 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac0014480 con 0x7faac815d600 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faad0672640 1 -- 172.21.15.163:0/2436651751 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac8152e40 con 0x7faac815d600 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faabeffd640 1 -- 172.21.15.163:0/2436651751 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0014870 con 0x7faac815d600 2024-05-08T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.233+0000 7faad0672640 1 -- 172.21.15.163:0/2436651751 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac8152660 con 0x7faac815d600 2024-05-08T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.234+0000 7faad0672640 1 -- 172.21.15.163:0/2436651751 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac806a550 con 0x7faac815d600 2024-05-08T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.235+0000 7faabeffd640 1 -- 172.21.15.163:0/2436651751 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faac0005d10 con 0x7faac815d600 2024-05-08T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.235+0000 7faabeffd640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7faaac0762d0 0x7faaac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.236+0000 7faace40e640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7faaac0762d0 0x7faaac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.236+0000 7faabeffd640 1 -- 172.21.15.163:0/2436651751 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7faac00981d0 con 0x7faac815d600 2024-05-08T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.236+0000 7faace40e640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7faaac0762d0 0x7faaac078790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faac80cf120 tx=0x7faac4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:39.238 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.240+0000 7faabeffd640 1 -- 172.21.15.163:0/2436651751 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faac0022070 con 0x7faac815d600 2024-05-08T06:37:39.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.423+0000 7faad0672640 1 -- 172.21.15.163:0/2436651751 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faac80cf780 con 0x7faac815d600 2024-05-08T06:37:39.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.424+0000 7faabeffd640 1 -- 172.21.15.163:0/2436651751 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16822 (secure 0 0 0) 0x7faac0010040 con 0x7faac815d600 2024-05-08T06:37:39.423 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:39.424 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":32,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:37:31.216129+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:37:20.131664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T06:37:21.318802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/880835577":"2024-05-09T06:37:31.216100+0000","172.21.15.163:6836/880835577":"2024-05-09T06:37:31.216100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:37:39.425 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.426+0000 7faad0672640 1 -- 172.21.15.163:0/2436651751 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7faaac0762d0 msgr2=0x7faaac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.426+0000 7faad0672640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7faaac0762d0 0x7faaac078790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faac80cf120 tx=0x7faac4009040 comp rx=0 tx=0).stop 2024-05-08T06:37:39.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.426+0000 7faad0672640 1 -- 172.21.15.163:0/2436651751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac815d600 msgr2=0x7faac815fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.426+0000 7faad0672640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac815d600 0x7faac815fd60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faac000efc0 tx=0x7faac000c490 comp rx=0 tx=0).stop 2024-05-08T06:37:39.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.426+0000 7faad0672640 1 -- 172.21.15.163:0/2436651751 shutdown_connections 2024-05-08T06:37:39.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.426+0000 7faad0672640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7faaac0762d0 0x7faaac078790 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.426+0000 7faad0672640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faac815d600 0x7faac815fd60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.426+0000 7faad0672640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faac806ef80 0x7faac8152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.426+0000 7faad0672640 1 --2- 172.21.15.163:0/2436651751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faac806da40 0x7faac8153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.426+0000 7faad0672640 1 -- 172.21.15.163:0/2436651751 >> 172.21.15.163:0/2436651751 conn(0x7faac8072ba0 msgr2=0x7faac80b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:39.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.426+0000 7faad0672640 1 -- 172.21.15.163:0/2436651751 shutdown_connections 2024-05-08T06:37:39.426 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.426+0000 7faad0672640 1 -- 172.21.15.163:0/2436651751 wait complete. 2024-05-08T06:37:39.437 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T06:37:39.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.538+0000 7f7741def640 1 Processor -- start 2024-05-08T06:37:39.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.538+0000 7f7741def640 1 -- start start 2024-05-08T06:37:39.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.539+0000 7f7741def640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f773c15f350 0x7f773c0bace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.539+0000 7f7741def640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f773c0b8240 0x7f773c0d3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.539+0000 7f7741def640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f773c0d2250 0x7f773c0d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.539+0000 7f7741def640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f773c0b9670 con 0x7f773c0b8240 2024-05-08T06:37:39.542 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.539+0000 7f7741def640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f773c0bb220 con 0x7f773c15f350 2024-05-08T06:37:39.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.539+0000 7f7741def640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f773c0b8a60 con 0x7f773c0d2250 2024-05-08T06:37:39.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.539+0000 7f773affd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f773c0b8240 0x7f773c0d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.539+0000 7f773affd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f773c0b8240 0x7f773c0d3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33784/0 (socket says 172.21.15.163:33784) 2024-05-08T06:37:39.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.540+0000 7f773affd640 1 -- 172.21.15.163:0/3837727294 learned_addr learned my addr 172.21.15.163:0/3837727294 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:39.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.540+0000 7f773b7fe640 1 --2- 172.21.15.163:0/3837727294 >> v2:172.21.15.192:3300/0 conn(0x7f773c15f350 0x7f773c0bace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.540+0000 7f773bfff640 1 --2- 172.21.15.163:0/3837727294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f773c0d2250 0x7f773c0d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.540+0000 7f773affd640 1 -- 172.21.15.163:0/3837727294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f773c0d2250 msgr2=0x7f773c0d26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:39.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.540+0000 7f773affd640 1 --2- 172.21.15.163:0/3837727294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f773c0d2250 0x7f773c0d26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.540+0000 7f773affd640 1 -- 172.21.15.163:0/3837727294 >> v2:172.21.15.192:3300/0 conn(0x7f773c15f350 msgr2=0x7f773c0bace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:39.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.540+0000 7f773affd640 1 --2- 172.21.15.163:0/3837727294 >> v2:172.21.15.192:3300/0 conn(0x7f773c15f350 0x7f773c0bace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.540+0000 7f773affd640 1 -- 172.21.15.163:0/3837727294 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773c0b8d40 con 0x7f773c0b8240 2024-05-08T06:37:39.544 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.540+0000 7f773bfff640 1 --2- 172.21.15.163:0/3837727294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f773c0d2250 0x7f773c0d26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:39.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.540+0000 7f773affd640 1 --2- 172.21.15.163:0/3837727294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f773c0b8240 0x7f773c0d3580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f773c06e370 tx=0x7f773002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705723c4107f5e22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:39.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.541+0000 7f773a7fc640 1 -- 172.21.15.163:0/3837727294 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730037070 con 0x7f773c0b8240 2024-05-08T06:37:39.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.541+0000 7f773a7fc640 1 -- 172.21.15.163:0/3837727294 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7730032440 con 0x7f773c0b8240 2024-05-08T06:37:39.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.541+0000 7f773a7fc640 1 -- 172.21.15.163:0/3837727294 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730032810 con 0x7f773c0b8240 2024-05-08T06:37:39.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.541+0000 7f7741def640 1 -- 172.21.15.163:0/3837727294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f773c0b8240 msgr2=0x7f773c0d3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.545 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.541+0000 7f7741def640 1 --2- 172.21.15.163:0/3837727294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f773c0b8240 0x7f773c0d3580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f773c06e370 tx=0x7f773002a270 comp rx=0 tx=0).stop 2024-05-08T06:37:39.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.541+0000 7f7741def640 1 -- 172.21.15.163:0/3837727294 shutdown_connections 2024-05-08T06:37:39.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.541+0000 7f7741def640 1 --2- 172.21.15.163:0/3837727294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f773c0d2250 0x7f773c0d26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.541+0000 7f7741def640 1 --2- 172.21.15.163:0/3837727294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f773c0b8240 0x7f773c0d3580 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.541+0000 7f7741def640 1 --2- 172.21.15.163:0/3837727294 >> v2:172.21.15.192:3300/0 conn(0x7f773c15f350 0x7f773c0bace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.541+0000 7f7741def640 1 -- 172.21.15.163:0/3837727294 >> 172.21.15.163:0/3837727294 conn(0x7f773c073e70 msgr2=0x7f773c074280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:39.546 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.541+0000 7f7741def640 1 -- 172.21.15.163:0/3837727294 shutdown_connections 2024-05-08T06:37:39.547 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.541+0000 7f7741def640 1 -- 172.21.15.163:0/3837727294 wait complete. 2024-05-08T06:37:39.547 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.542+0000 7f7741def640 1 Processor -- start 2024-05-08T06:37:39.547 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.543+0000 7f7741def640 1 -- start start 2024-05-08T06:37:39.551 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.543+0000 7f7741def640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f773c0b8240 0x7f773c1516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.543+0000 7f7741def640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f773c0d2250 0x7f773c0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.543+0000 7f7741def640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f773c15f350 0x7f773c161760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.543+0000 7f7741def640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f773c0d3ac0 con 0x7f773c15f350 2024-05-08T06:37:39.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.543+0000 7f7741def640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f773c0b9300 con 0x7f773c0b8240 2024-05-08T06:37:39.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.543+0000 7f7741def640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f773c15f730 con 0x7f773c0d2250 2024-05-08T06:37:39.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.543+0000 7f773bfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f773c15f350 0x7f773c161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f773bfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f773c15f350 0x7f773c161760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33792/0 (socket says 172.21.15.163:33792) 2024-05-08T06:37:39.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f773bfff640 1 -- 172.21.15.163:0/408678493 learned_addr learned my addr 172.21.15.163:0/408678493 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:39.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f773affd640 1 --2- 172.21.15.163:0/408678493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f773c0d2250 0x7f773c0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f773b7fe640 1 --2- 172.21.15.163:0/408678493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f773c0b8240 0x7f773c1516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f773bfff640 1 -- 172.21.15.163:0/408678493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f773c0d2250 msgr2=0x7f773c0cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f773bfff640 1 --2- 172.21.15.163:0/408678493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f773c0d2250 0x7f773c0cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f773bfff640 1 -- 172.21.15.163:0/408678493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f773c0b8240 msgr2=0x7f773c1516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f773bfff640 1 --2- 172.21.15.163:0/408678493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f773c0b8240 0x7f773c1516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f773bfff640 1 -- 172.21.15.163:0/408678493 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773c151c20 con 0x7f773c15f350 2024-05-08T06:37:39.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f773bfff640 1 --2- 172.21.15.163:0/408678493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f773c15f350 0x7f773c161760 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f773400e5e0 tx=0x7f773400eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:39.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f7723fff640 1 -- 172.21.15.163:0/408678493 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773400f040 con 0x7f773c15f350 2024-05-08T06:37:39.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f7723fff640 1 -- 172.21.15.163:0/408678493 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7734004230 con 0x7f773c15f350 2024-05-08T06:37:39.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f7741def640 1 -- 172.21.15.163:0/408678493 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773c1e8570 con 0x7f773c15f350 2024-05-08T06:37:39.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f7723fff640 1 -- 172.21.15.163:0/408678493 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734003610 con 0x7f773c15f350 2024-05-08T06:37:39.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.544+0000 7f7741def640 1 -- 172.21.15.163:0/408678493 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773c1e8b30 con 0x7f773c15f350 2024-05-08T06:37:39.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.545+0000 7f7741def640 1 -- 172.21.15.163:0/408678493 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f773c06a4b0 con 0x7f773c15f350 2024-05-08T06:37:39.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.546+0000 7f7723fff640 1 -- 172.21.15.163:0/408678493 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f77340037b0 con 0x7f773c15f350 2024-05-08T06:37:39.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.546+0000 7f7723fff640 1 --2- 172.21.15.163:0/408678493 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f77280762d0 0x7f7728078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.546+0000 7f7723fff640 1 -- 172.21.15.163:0/408678493 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f7734097190 con 0x7f773c15f350 2024-05-08T06:37:39.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.546+0000 7f773b7fe640 1 --2- 172.21.15.163:0/408678493 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f77280762d0 0x7f7728078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.547+0000 7f773b7fe640 1 --2- 172.21.15.163:0/408678493 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f77280762d0 0x7f7728078790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f772c0113e0 tx=0x7f772c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:39.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.550+0000 7f7723fff640 1 -- 172.21.15.163:0/408678493 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f773c151c20 con 0x7f773c15f350 2024-05-08T06:37:39.769 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.770+0000 7f7741def640 1 -- 172.21.15.163:0/408678493 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f773c0b5950 con 0x7f773c15f350 2024-05-08T06:37:39.770 INFO:tasks.ceph.mon.a.smithi163.stderr:2024-05-08T06:37:39.771+0000 7fa54ba64640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:37:39.771 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.772+0000 7f7723fff640 1 -- 172.21.15.163:0/408678493 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7f7734017020 con 0x7f773c15f350 2024-05-08T06:37:39.771 INFO:teuthology.orchestra.run.smithi163.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T06:37:39.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.775+0000 7f7741def640 1 -- 172.21.15.163:0/408678493 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f77280762d0 msgr2=0x7f7728078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.775+0000 7f7741def640 1 --2- 172.21.15.163:0/408678493 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f77280762d0 0x7f7728078790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f772c0113e0 tx=0x7f772c00b040 comp rx=0 tx=0).stop 2024-05-08T06:37:39.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.775+0000 7f7741def640 1 -- 172.21.15.163:0/408678493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f773c15f350 msgr2=0x7f773c161760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.775+0000 7f7741def640 1 --2- 172.21.15.163:0/408678493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f773c15f350 0x7f773c161760 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f773400e5e0 tx=0x7f773400eaa0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.775+0000 7f7741def640 1 -- 172.21.15.163:0/408678493 shutdown_connections 2024-05-08T06:37:39.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.775+0000 7f7741def640 1 --2- 172.21.15.163:0/408678493 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f77280762d0 0x7f7728078790 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.775+0000 7f7741def640 1 --2- 172.21.15.163:0/408678493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f773c15f350 0x7f773c161760 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.774 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.775+0000 7f7741def640 1 --2- 172.21.15.163:0/408678493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f773c0d2250 0x7f773c0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.775+0000 7f7741def640 1 --2- 172.21.15.163:0/408678493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f773c0b8240 0x7f773c1516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.775+0000 7f7741def640 1 -- 172.21.15.163:0/408678493 >> 172.21.15.163:0/408678493 conn(0x7f773c073e70 msgr2=0x7f773c0d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:39.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.775+0000 7f7741def640 1 -- 172.21.15.163:0/408678493 shutdown_connections 2024-05-08T06:37:39.775 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.775+0000 7f7741def640 1 -- 172.21.15.163:0/408678493 wait complete. 2024-05-08T06:37:39.776 INFO:tasks.ceph.mds.a.smithi163.stderr: -92> 2024-05-08T06:37:37.227+0000 7f666b0ba640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T06:37:39.787 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T06:37:39.802 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2024-05-08T06:37:39.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.890+0000 7f3cdfe74640 1 Processor -- start 2024-05-08T06:37:39.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.890+0000 7f3cdfe74640 1 -- start start 2024-05-08T06:37:39.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.891+0000 7f3cdfe74640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd81603f0 0x7f3cd81607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.891+0000 7f3cdfe74640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3cd80b5f10 0x7f3cd815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.891+0000 7f3cdfe74640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cd80b4e70 con 0x7f3cd815d3b0 2024-05-08T06:37:39.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.891+0000 7f3cdfe74640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3cd80b6370 con 0x7f3cd80b5f10 2024-05-08T06:37:39.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.891+0000 7f3cdfe74640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd80b5150 con 0x7f3cd81603f0 2024-05-08T06:37:39.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.891+0000 7f3cde411640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3cd815d3b0 0x7f3cd80d1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:41602/0 (socket says 172.21.15.163:41602) 2024-05-08T06:37:39.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.891+0000 7f3cde411640 1 -- 172.21.15.163:0/2315299200 learned_addr learned my addr 172.21.15.163:0/2315299200 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:39.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.891+0000 7f3cdd40f640 1 --2- 172.21.15.163:0/2315299200 >> v2:172.21.15.192:3300/0 conn(0x7f3cd80b5f10 0x7f3cd815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.891+0000 7f3cddc10640 1 --2- 172.21.15.163:0/2315299200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd81603f0 0x7f3cd81607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.891+0000 7f3cdcc0e640 1 -- 172.21.15.163:0/2315299200 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1976677794 0 0) 0x7f3cd80b4e70 con 0x7f3cd815d3b0 2024-05-08T06:37:39.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.891+0000 7f3cdcc0e640 1 -- 172.21.15.163:0/2315299200 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cc4005730 con 0x7f3cd815d3b0 2024-05-08T06:37:39.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdcc0e640 1 -- 172.21.15.163:0/2315299200 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 57976730 0 0) 0x7f3cc4005730 con 0x7f3cd815d3b0 2024-05-08T06:37:39.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdcc0e640 1 -- 172.21.15.163:0/2315299200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd81603f0 msgr2=0x7f3cd81607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:39.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdcc0e640 1 --2- 172.21.15.163:0/2315299200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd81603f0 0x7f3cd81607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdcc0e640 1 -- 172.21.15.163:0/2315299200 >> v2:172.21.15.192:3300/0 conn(0x7f3cd80b5f10 msgr2=0x7f3cd815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:39.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdcc0e640 1 --2- 172.21.15.163:0/2315299200 >> v2:172.21.15.192:3300/0 conn(0x7f3cd80b5f10 0x7f3cd815ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdcc0e640 1 -- 172.21.15.163:0/2315299200 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd80b5430 con 0x7f3cd815d3b0 2024-05-08T06:37:39.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdcc0e640 1 -- 172.21.15.163:0/2315299200 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f3cd00016f0 con 0x7f3cd815d3b0 2024-05-08T06:37:39.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cddc10640 1 --2- 172.21.15.163:0/2315299200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd81603f0 0x7f3cd81607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:39.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdcc0e640 1 -- 172.21.15.163:0/2315299200 >> v1:172.21.15.163:6789/0 conn(0x7f3cd815d3b0 legacy=0x7f3cd80d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdcc0e640 1 --2- 172.21.15.163:0/2315299200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc4005c10 0x7f3cc4006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdcc0e640 1 --2- 172.21.15.163:0/2315299200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd81603f0 0x7f3cc4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdcc0e640 1 --2- 172.21.15.163:0/2315299200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cc4006f50 0x7f3cc40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdcc0e640 1 -- 172.21.15.163:0/2315299200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd80b6370 con 0x7f3cc4005c10 2024-05-08T06:37:39.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdcc0e640 1 -- 172.21.15.163:0/2315299200 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd80b5150 con 0x7f3cc4006f50 2024-05-08T06:37:39.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdcc0e640 1 -- 172.21.15.163:0/2315299200 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc400dbd0 con 0x7f3cd81603f0 2024-05-08T06:37:39.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdd40f640 1 --2- 172.21.15.163:0/2315299200 >> v2:172.21.15.192:3300/0 conn(0x7f3cd80b5f10 0x7f3cd815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:39.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cddc10640 1 --2- 172.21.15.163:0/2315299200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc4005c10 0x7f3cc4006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cde411640 1 --2- 172.21.15.163:0/2315299200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cc4006f50 0x7f3cc40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdd40f640 1 --2- 172.21.15.163:0/2315299200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd81603f0 0x7f3cc4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cddc10640 1 -- 172.21.15.163:0/2315299200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd81603f0 msgr2=0x7f3cc4006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:39.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cddc10640 1 --2- 172.21.15.163:0/2315299200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd81603f0 0x7f3cc4006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cddc10640 1 -- 172.21.15.163:0/2315299200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cc4006f50 msgr2=0x7f3cc40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:39.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cddc10640 1 --2- 172.21.15.163:0/2315299200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cc4006f50 0x7f3cc40073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cddc10640 1 -- 172.21.15.163:0/2315299200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc4005730 con 0x7f3cc4005c10 2024-05-08T06:37:39.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.892+0000 7f3cdd40f640 1 --2- 172.21.15.163:0/2315299200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd81603f0 0x7f3cc4006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:39.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.893+0000 7f3cddc10640 1 --2- 172.21.15.163:0/2315299200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc4005c10 0x7f3cc4006030 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400e350 tx=0x7f3cc4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab4f19ceff4f145 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:39.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.893+0000 7f3cdcc0e640 1 -- 172.21.15.163:0/2315299200 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4033070 con 0x7f3cc4005c10 2024-05-08T06:37:39.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.893+0000 7f3cdcc0e640 1 -- 172.21.15.163:0/2315299200 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd402f660 con 0x7f3cc4005c10 2024-05-08T06:37:39.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.893+0000 7f3cdcc0e640 1 -- 172.21.15.163:0/2315299200 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4009de0 con 0x7f3cc4005c10 2024-05-08T06:37:39.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.894+0000 7f3cdfe74640 1 -- 172.21.15.163:0/2315299200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc4005c10 msgr2=0x7f3cc4006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.894+0000 7f3cdfe74640 1 --2- 172.21.15.163:0/2315299200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc4005c10 0x7f3cc4006030 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400e350 tx=0x7f3cc4007a40 comp rx=0 tx=0).stop 2024-05-08T06:37:39.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.894+0000 7f3cde411640 1 -- 172.21.15.163:0/2315299200 reap_dead start 2024-05-08T06:37:39.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.894+0000 7f3cdfe74640 1 -- 172.21.15.163:0/2315299200 shutdown_connections 2024-05-08T06:37:39.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.894+0000 7f3cdfe74640 1 -- 172.21.15.163:0/2315299200 >> 172.21.15.163:0/2315299200 conn(0x7f3cd806c500 msgr2=0x7f3cd806c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:39.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.894+0000 7f3cdfe74640 1 -- 172.21.15.163:0/2315299200 shutdown_connections 2024-05-08T06:37:39.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.894+0000 7f3cdfe74640 1 -- 172.21.15.163:0/2315299200 wait complete. 2024-05-08T06:37:39.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.895+0000 7f3cdfe74640 1 Processor -- start 2024-05-08T06:37:39.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.895+0000 7f3cdfe74640 1 -- start start 2024-05-08T06:37:39.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdfe74640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cc4005c10 0x7f3cd814ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdfe74640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cd814d1c0 0x7f3cd814d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdfe74640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd80afb90 0x7f3cd80afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdfe74640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd80d1980 con 0x7f3cd814d1c0 2024-05-08T06:37:39.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdfe74640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd80b59f0 con 0x7f3cc4005c10 2024-05-08T06:37:39.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdfe74640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd80d1b00 con 0x7f3cd80afb90 2024-05-08T06:37:39.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdd40f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cd814d1c0 0x7f3cd814d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdd40f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cd814d1c0 0x7f3cd814d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33812/0 (socket says 172.21.15.163:33812) 2024-05-08T06:37:39.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdd40f640 1 -- 172.21.15.163:0/3660741124 learned_addr learned my addr 172.21.15.163:0/3660741124 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:39.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cddc10640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cc4005c10 0x7f3cd814ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cde411640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd80afb90 0x7f3cd80afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdd40f640 1 -- 172.21.15.163:0/3660741124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd80afb90 msgr2=0x7f3cd80afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdd40f640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd80afb90 0x7f3cd80afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdd40f640 1 -- 172.21.15.163:0/3660741124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cc4005c10 msgr2=0x7f3cd814ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:39.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdd40f640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cc4005c10 0x7f3cd814ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:39.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdd40f640 1 -- 172.21.15.163:0/3660741124 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd81f5290 con 0x7f3cd814d1c0 2024-05-08T06:37:39.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cddc10640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cc4005c10 0x7f3cd814ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:39.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdd40f640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cd814d1c0 0x7f3cd814d640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80118d0 tx=0x7f3cc8011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:39.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cce7fc640 1 -- 172.21.15.163:0/3660741124 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc800fbd0 con 0x7f3cd814d1c0 2024-05-08T06:37:39.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cce7fc640 1 -- 172.21.15.163:0/3660741124 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc800fd70 con 0x7f3cd814d1c0 2024-05-08T06:37:39.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdfe74640 1 -- 172.21.15.163:0/3660741124 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd814cf80 con 0x7f3cd814d1c0 2024-05-08T06:37:39.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cce7fc640 1 -- 172.21.15.163:0/3660741124 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8015680 con 0x7f3cd814d1c0 2024-05-08T06:37:39.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.896+0000 7f3cdfe74640 1 -- 172.21.15.163:0/3660741124 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd81f5810 con 0x7f3cd814d1c0 2024-05-08T06:37:39.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.897+0000 7f3cdfe74640 1 -- 172.21.15.163:0/3660741124 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd806a550 con 0x7f3cd814d1c0 2024-05-08T06:37:39.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.898+0000 7f3cce7fc640 1 -- 172.21.15.163:0/3660741124 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3cc8015820 con 0x7f3cd814d1c0 2024-05-08T06:37:39.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.899+0000 7f3cce7fc640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f3cbc0762d0 0x7f3cbc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:39.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.899+0000 7f3cce7fc640 1 -- 172.21.15.163:0/3660741124 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5881+0+0 (secure 0 0 0) 0x7f3cc80981b0 con 0x7f3cd814d1c0 2024-05-08T06:37:39.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.899+0000 7f3cddc10640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f3cbc0762d0 0x7f3cbc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:39.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.899+0000 7f3cddc10640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f3cbc0762d0 0x7f3cbc078790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4035000 tx=0x7f3cd402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:39.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:39.903+0000 7f3cce7fc640 1 -- 172.21.15.163:0/3660741124 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3cc8019030 con 0x7f3cd814d1c0 2024-05-08T06:37:40.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.126+0000 7f3cdfe74640 1 -- 172.21.15.163:0/3660741124 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3cd80b4bb0 con 0x7f3cd814d1c0 2024-05-08T06:37:40.777 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.778+0000 7f3cce7fc640 1 -- 172.21.15.163:0/3660741124 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7f3cc8061b20 con 0x7f3cd814d1c0 2024-05-08T06:37:40.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.780+0000 7f3cdfe74640 1 -- 172.21.15.163:0/3660741124 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f3cbc0762d0 msgr2=0x7f3cbc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:40.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.780+0000 7f3cdfe74640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f3cbc0762d0 0x7f3cbc078790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4035000 tx=0x7f3cd402f000 comp rx=0 tx=0).stop 2024-05-08T06:37:40.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.780+0000 7f3cdfe74640 1 -- 172.21.15.163:0/3660741124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cd814d1c0 msgr2=0x7f3cd814d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:40.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.780+0000 7f3cdfe74640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cd814d1c0 0x7f3cd814d640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80118d0 tx=0x7f3cc8011d90 comp rx=0 tx=0).stop 2024-05-08T06:37:40.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.780+0000 7f3cdfe74640 1 -- 172.21.15.163:0/3660741124 shutdown_connections 2024-05-08T06:37:40.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.780+0000 7f3cdfe74640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f3cbc0762d0 0x7f3cbc078790 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:40.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.780+0000 7f3cdfe74640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cd80afb90 0x7f3cd80afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:40.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.780+0000 7f3cdfe74640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cd814d1c0 0x7f3cd814d640 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:40.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.780+0000 7f3cdfe74640 1 --2- 172.21.15.163:0/3660741124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cc4005c10 0x7f3cd814ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:40.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.780+0000 7f3cdfe74640 1 -- 172.21.15.163:0/3660741124 >> 172.21.15.163:0/3660741124 conn(0x7f3cd806c500 msgr2=0x7f3cd80d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:40.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.780+0000 7f3cdfe74640 1 -- 172.21.15.163:0/3660741124 shutdown_connections 2024-05-08T06:37:40.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.780+0000 7f3cdfe74640 1 -- 172.21.15.163:0/3660741124 wait complete. 2024-05-08T06:37:40.792 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T06:37:40.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.939+0000 7fe1dc1a4640 1 Processor -- start 2024-05-08T06:37:40.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.939+0000 7fe1dc1a4640 1 -- start start 2024-05-08T06:37:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.940+0000 7fe1dc1a4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe1d415c120 0x7fe1d406fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.940+0000 7fe1dc1a4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe1d40af9b0 0x7fe1d415fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.940+0000 7fe1dc1a4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1d415c500 con 0x7fe1d40b0390 2024-05-08T06:37:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.940+0000 7fe1dc1a4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d40afe90 con 0x7fe1d40af9b0 2024-05-08T06:37:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.940+0000 7fe1dc1a4640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d40b49d0 con 0x7fe1d415c120 2024-05-08T06:37:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.940+0000 7fe1da741640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe1d40af9b0 0x7fe1d415fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.940+0000 7fe1d973f640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe1d40b0390 0x7fe1d40b54b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:41618/0 (socket says 172.21.15.163:41618) 2024-05-08T06:37:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.940+0000 7fe1d973f640 1 -- 172.21.15.163:0/1984015128 learned_addr learned my addr 172.21.15.163:0/1984015128 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.940+0000 7fe1da741640 1 -- 172.21.15.163:0/1984015128 >> v2:172.21.15.192:3300/0 conn(0x7fe1d415c120 msgr2=0x7fe1d406fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.940+0000 7fe1da741640 1 --2- 172.21.15.163:0/1984015128 >> v2:172.21.15.192:3300/0 conn(0x7fe1d415c120 0x7fe1d406fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.940+0000 7fe1da741640 1 -- 172.21.15.163:0/1984015128 >> v1:172.21.15.163:6789/0 conn(0x7fe1d40b0390 legacy=0x7fe1d40b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.940+0000 7fe1da741640 1 -- 172.21.15.163:0/1984015128 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d40b4cb0 con 0x7fe1d40af9b0 2024-05-08T06:37:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.941+0000 7fe1da741640 1 --2- 172.21.15.163:0/1984015128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d40af9b0 0x7fe1d415fc60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4070450 tx=0x7fe1cc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804ef9cb80a9a67b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.941+0000 7fe1d8f3e640 1 -- 172.21.15.163:0/1984015128 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc007740 con 0x7fe1d40af9b0 2024-05-08T06:37:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.941+0000 7fe1d8f3e640 1 -- 172.21.15.163:0/1984015128 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1cc0078e0 con 0x7fe1d40af9b0 2024-05-08T06:37:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.941+0000 7fe1d8f3e640 1 -- 172.21.15.163:0/1984015128 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc007cb0 con 0x7fe1d40af9b0 2024-05-08T06:37:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.942+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/1984015128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d40af9b0 msgr2=0x7fe1d415fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.942+0000 7fe1dc1a4640 1 --2- 172.21.15.163:0/1984015128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d40af9b0 0x7fe1d415fc60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4070450 tx=0x7fe1cc02a5f0 comp rx=0 tx=0).stop 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.942+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/1984015128 shutdown_connections 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.942+0000 7fe1dc1a4640 1 --2- 172.21.15.163:0/1984015128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d40af9b0 0x7fe1d415fc60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.942+0000 7fe1dc1a4640 1 --2- 172.21.15.163:0/1984015128 >> v2:172.21.15.192:3300/0 conn(0x7fe1d415c120 0x7fe1d406fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.942+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/1984015128 >> 172.21.15.163:0/1984015128 conn(0x7fe1d40b2b40 msgr2=0x7fe1d40b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.942+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/1984015128 shutdown_connections 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.942+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/1984015128 wait complete. 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.943+0000 7fe1dc1a4640 1 Processor -- start 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.943+0000 7fe1dc1a4640 1 -- start start 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1dc1a4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1d40af9b0 0x7fe1d4158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1dc1a4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d40b0390 0x7fe1d415eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1dc1a4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1d415c120 0x7fe1d41e5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1dc1a4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d41601a0 con 0x7fe1d40b0390 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1dc1a4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d40b5bc0 con 0x7fe1d40af9b0 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1dc1a4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d4160320 con 0x7fe1d415c120 2024-05-08T06:37:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1d973f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d40b0390 0x7fe1d415eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1d973f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d40b0390 0x7fe1d415eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33816/0 (socket says 172.21.15.163:33816) 2024-05-08T06:37:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1d973f640 1 -- 172.21.15.163:0/974497122 learned_addr learned my addr 172.21.15.163:0/974497122 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1d9f40640 1 --2- 172.21.15.163:0/974497122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1d40af9b0 0x7fe1d4158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1da741640 1 --2- 172.21.15.163:0/974497122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1d415c120 0x7fe1d41e5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1d973f640 1 -- 172.21.15.163:0/974497122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1d415c120 msgr2=0x7fe1d41e5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1d973f640 1 --2- 172.21.15.163:0/974497122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1d415c120 0x7fe1d41e5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1d973f640 1 -- 172.21.15.163:0/974497122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1d40af9b0 msgr2=0x7fe1d4158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1d973f640 1 --2- 172.21.15.163:0/974497122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1d40af9b0 0x7fe1d4158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1d973f640 1 -- 172.21.15.163:0/974497122 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d4159110 con 0x7fe1d40b0390 2024-05-08T06:37:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1d973f640 1 --2- 172.21.15.163:0/974497122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d40b0390 0x7fe1d415eb80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400c500 tx=0x7fe1c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.944+0000 7fe1ca7fc640 1 -- 172.21.15.163:0/974497122 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4017070 con 0x7fe1d40b0390 2024-05-08T06:37:40.947 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.945+0000 7fe1ca7fc640 1 -- 172.21.15.163:0/974497122 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c4005d00 con 0x7fe1d40b0390 2024-05-08T06:37:40.947 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.945+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/974497122 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d4157b30 con 0x7fe1d40b0390 2024-05-08T06:37:40.947 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.945+0000 7fe1ca7fc640 1 -- 172.21.15.163:0/974497122 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4007330 con 0x7fe1d40b0390 2024-05-08T06:37:40.947 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.945+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/974497122 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d41580f0 con 0x7fe1d40b0390 2024-05-08T06:37:40.947 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.946+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/974497122 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d406a550 con 0x7fe1d40b0390 2024-05-08T06:37:40.948 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.946+0000 7fe1ca7fc640 1 -- 172.21.15.163:0/974497122 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe1c401a070 con 0x7fe1d40b0390 2024-05-08T06:37:40.948 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.947+0000 7fe1ca7fc640 1 --2- 172.21.15.163:0/974497122 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe1b80762d0 0x7fe1b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:40.948 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.947+0000 7fe1ca7fc640 1 -- 172.21.15.163:0/974497122 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5881+0+0 (secure 0 0 0) 0x7fe1c4096940 con 0x7fe1d40b0390 2024-05-08T06:37:40.948 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.947+0000 7fe1d9f40640 1 --2- 172.21.15.163:0/974497122 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe1b80762d0 0x7fe1b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:40.948 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.947+0000 7fe1d9f40640 1 --2- 172.21.15.163:0/974497122 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe1b80762d0 0x7fe1b8078790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe1d41f6900 tx=0x7fe1d0009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:40.950 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:40.951+0000 7fe1ca7fc640 1 -- 172.21.15.163:0/974497122 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe1c405fc30 con 0x7fe1d40b0390 2024-05-08T06:37:41.130 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.131+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/974497122 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe1d406e5c0 con 0x7fe1d40b0390 2024-05-08T06:37:41.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.779+0000 7fe1ca7fc640 1 -- 172.21.15.163:0/974497122 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) ==== 181+0+0 (secure 0 0 0) 0x7fe1c4063f30 con 0x7fe1d40b0390 2024-05-08T06:37:41.779 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' removed 2024-05-08T06:37:41.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.784+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/974497122 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe1b80762d0 msgr2=0x7fe1b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:41.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.784+0000 7fe1dc1a4640 1 --2- 172.21.15.163:0/974497122 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe1b80762d0 0x7fe1b8078790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe1d41f6900 tx=0x7fe1d0009060 comp rx=0 tx=0).stop 2024-05-08T06:37:41.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.784+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/974497122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d40b0390 msgr2=0x7fe1d415eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:41.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.784+0000 7fe1dc1a4640 1 --2- 172.21.15.163:0/974497122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d40b0390 0x7fe1d415eb80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400c500 tx=0x7fe1c400c9c0 comp rx=0 tx=0).stop 2024-05-08T06:37:41.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.784+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/974497122 shutdown_connections 2024-05-08T06:37:41.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.784+0000 7fe1dc1a4640 1 --2- 172.21.15.163:0/974497122 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fe1b80762d0 0x7fe1b8078790 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe1d41f6900 tx=0x7fe1d0009060 comp rx=0 tx=0).stop 2024-05-08T06:37:41.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.784+0000 7fe1dc1a4640 1 --2- 172.21.15.163:0/974497122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1d415c120 0x7fe1d41e5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:41.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.784+0000 7fe1dc1a4640 1 --2- 172.21.15.163:0/974497122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d40b0390 0x7fe1d415eb80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:41.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.784+0000 7fe1dc1a4640 1 --2- 172.21.15.163:0/974497122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1d40af9b0 0x7fe1d4158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:41.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.784+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/974497122 >> 172.21.15.163:0/974497122 conn(0x7fe1d40b2b40 msgr2=0x7fe1d4159bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:41.785 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.784+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/974497122 shutdown_connections 2024-05-08T06:37:41.785 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.784+0000 7fe1dc1a4640 1 -- 172.21.15.163:0/974497122 wait complete. 2024-05-08T06:37:41.796 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T06:37:41.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.903+0000 7fca08c83640 1 Processor -- start 2024-05-08T06:37:41.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.903+0000 7fca08c83640 1 -- start start 2024-05-08T06:37:41.906 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.904+0000 7fca08c83640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fca0406cd30 0x7fca040af010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:41.906 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.904+0000 7fca08c83640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca04163020 con 0x7fca040b2c90 2024-05-08T06:37:41.906 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.904+0000 7fca08c83640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca0406d1c0 con 0x7fca04162c40 2024-05-08T06:37:41.906 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.904+0000 7fca08c83640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fca040b1cb0 con 0x7fca0406cd30 2024-05-08T06:37:41.906 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.904+0000 7fca02d9d640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fca040b2c90 0x7fca040d0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:41622/0 (socket says 172.21.15.163:41622) 2024-05-08T06:37:41.906 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.904+0000 7fca02d9d640 1 -- 172.21.15.163:0/2005214072 learned_addr learned my addr 172.21.15.163:0/2005214072 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:41.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.904+0000 7fca01d9b640 1 --2- 172.21.15.163:0/2005214072 >> v2:172.21.15.192:3300/0 conn(0x7fca0406cd30 0x7fca040af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:41.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.904+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3965610298 0 0) 0x7fca04163020 con 0x7fca040b2c90 2024-05-08T06:37:41.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.904+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9ec003520 con 0x7fca040b2c90 2024-05-08T06:37:41.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3466154812 0 0) 0x7fca0406d1c0 con 0x7fca04162c40 2024-05-08T06:37:41.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca04163020 con 0x7fca04162c40 2024-05-08T06:37:41.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3812270838 0 0) 0x7fc9ec003520 con 0x7fca040b2c90 2024-05-08T06:37:41.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 >> v2:172.21.15.192:3300/0 conn(0x7fca0406cd30 msgr2=0x7fca040af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:41.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 --2- 172.21.15.163:0/2005214072 >> v2:172.21.15.192:3300/0 conn(0x7fca0406cd30 0x7fca040af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:41.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 >> v1:172.21.15.192:6789/0 conn(0x7fca04162c40 legacy=0x7fca040b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:41.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca040b1f90 con 0x7fca040b2c90 2024-05-08T06:37:41.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7fc9fc001a70 con 0x7fca040b2c90 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 >> v1:172.21.15.163:6789/0 conn(0x7fca040b2c90 legacy=0x7fca040d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 --2- 172.21.15.163:0/2005214072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9ec003c30 0x7fc9ec004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 --2- 172.21.15.163:0/2005214072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ec004800 0x7fc9ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 --2- 172.21.15.163:0/2005214072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9ec005210 0x7fc9ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fca040b1cb0 con 0x7fc9ec005210 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec00c090 con 0x7fc9ec003c30 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc9ec00c290 con 0x7fc9ec004800 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca02d9d640 1 --2- 172.21.15.163:0/2005214072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9ec005210 0x7fc9ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca0259c640 1 --2- 172.21.15.163:0/2005214072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9ec003c30 0x7fc9ec004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca01d9b640 1 --2- 172.21.15.163:0/2005214072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ec004800 0x7fc9ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca02d9d640 1 -- 172.21.15.163:0/2005214072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ec004800 msgr2=0x7fc9ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca02d9d640 1 --2- 172.21.15.163:0/2005214072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ec004800 0x7fc9ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca02d9d640 1 -- 172.21.15.163:0/2005214072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9ec003c30 msgr2=0x7fc9ec004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca02d9d640 1 --2- 172.21.15.163:0/2005214072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9ec003c30 0x7fc9ec004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:41.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca02d9d640 1 -- 172.21.15.163:0/2005214072 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ec003520 con 0x7fc9ec005210 2024-05-08T06:37:41.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.905+0000 7fca01d9b640 1 --2- 172.21.15.163:0/2005214072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ec004800 0x7fc9ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:41.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.906+0000 7fca02d9d640 1 -- 172.21.15.163:0/2005214072 reap_dead start 2024-05-08T06:37:41.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.906+0000 7fca02d9d640 1 --2- 172.21.15.163:0/2005214072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9ec005210 0x7fc9ec00b9c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fca04164380 tx=0x7fc9ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f545a52f8b95a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:41.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.906+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc00a430 con 0x7fc9ec005210 2024-05-08T06:37:41.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.906+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9fc02f070 con 0x7fc9ec005210 2024-05-08T06:37:41.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.906+0000 7fca0159a640 1 -- 172.21.15.163:0/2005214072 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc02bb40 con 0x7fc9ec005210 2024-05-08T06:37:41.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.906+0000 7fca08c83640 1 -- 172.21.15.163:0/2005214072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9ec005210 msgr2=0x7fc9ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:41.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.906+0000 7fca08c83640 1 --2- 172.21.15.163:0/2005214072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9ec005210 0x7fc9ec00b9c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fca04164380 tx=0x7fc9ec005830 comp rx=0 tx=0).stop 2024-05-08T06:37:41.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.907+0000 7fca08c83640 1 -- 172.21.15.163:0/2005214072 shutdown_connections 2024-05-08T06:37:41.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.907+0000 7fca08c83640 1 --2- 172.21.15.163:0/2005214072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9ec005210 0x7fc9ec00b9c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:41.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.907+0000 7fca08c83640 1 -- 172.21.15.163:0/2005214072 >> 172.21.15.163:0/2005214072 conn(0x7fca040739a0 msgr2=0x7fca040b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:41.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.907+0000 7fca08c83640 1 -- 172.21.15.163:0/2005214072 shutdown_connections 2024-05-08T06:37:41.910 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.907+0000 7fca08c83640 1 -- 172.21.15.163:0/2005214072 wait complete. 2024-05-08T06:37:41.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.908+0000 7fca08c83640 1 Processor -- start 2024-05-08T06:37:41.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.908+0000 7fca08c83640 1 -- start start 2024-05-08T06:37:41.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca08c83640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ec005210 0x7fca04159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:41.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca08c83640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca041f6180 0x7fca04070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:41.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca08c83640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca041f6600 0x7fca04070c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:41.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca08c83640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fca040b4a80 con 0x7fca041f6180 2024-05-08T06:37:41.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca08c83640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fca040d0800 con 0x7fca041f6600 2024-05-08T06:37:41.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca08c83640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fca040b4c00 con 0x7fc9ec005210 2024-05-08T06:37:41.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca01d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca041f6180 0x7fca04070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:41.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca01d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca041f6180 0x7fca04070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33842/0 (socket says 172.21.15.163:33842) 2024-05-08T06:37:41.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca01d9b640 1 -- 172.21.15.163:0/259212813 learned_addr learned my addr 172.21.15.163:0/259212813 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:41.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca02d9d640 1 --2- 172.21.15.163:0/259212813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca041f6600 0x7fca04070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:41.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca0259c640 1 --2- 172.21.15.163:0/259212813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ec005210 0x7fca04159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:41.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca01d9b640 1 -- 172.21.15.163:0/259212813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ec005210 msgr2=0x7fca04159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:41.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca01d9b640 1 --2- 172.21.15.163:0/259212813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ec005210 0x7fca04159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:41.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca01d9b640 1 -- 172.21.15.163:0/259212813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca041f6600 msgr2=0x7fca04070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:41.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca01d9b640 1 --2- 172.21.15.163:0/259212813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca041f6600 0x7fca04070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:41.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca01d9b640 1 -- 172.21.15.163:0/259212813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0415fc10 con 0x7fca041f6180 2024-05-08T06:37:41.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca01d9b640 1 --2- 172.21.15.163:0/259212813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca041f6180 0x7fca04070700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8011570 tx=0x7fc9f8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:41.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fc9f2ffd640 1 -- 172.21.15.163:0/259212813 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8015870 con 0x7fca041f6180 2024-05-08T06:37:41.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fc9f2ffd640 1 -- 172.21.15.163:0/259212813 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9f8015a10 con 0x7fca041f6180 2024-05-08T06:37:41.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fc9f2ffd640 1 -- 172.21.15.163:0/259212813 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8015dc0 con 0x7fca041f6180 2024-05-08T06:37:41.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca08c83640 1 -- 172.21.15.163:0/259212813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca041f9240 con 0x7fca041f6180 2024-05-08T06:37:41.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.909+0000 7fca08c83640 1 -- 172.21.15.163:0/259212813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca0415a920 con 0x7fca041f6180 2024-05-08T06:37:41.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.910+0000 7fca08c83640 1 -- 172.21.15.163:0/259212813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca040d0370 con 0x7fca041f6180 2024-05-08T06:37:41.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.911+0000 7fc9f2ffd640 1 -- 172.21.15.163:0/259212813 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc9f800f460 con 0x7fca041f6180 2024-05-08T06:37:41.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.912+0000 7fc9f2ffd640 1 --2- 172.21.15.163:0/259212813 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc9e40762d0 0x7fc9e4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:41.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.912+0000 7fc9f2ffd640 1 -- 172.21.15.163:0/259212813 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5443+0+0 (secure 0 0 0) 0x7fc9f8097d30 con 0x7fca041f6180 2024-05-08T06:37:41.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.912+0000 7fca0259c640 1 --2- 172.21.15.163:0/259212813 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc9e40762d0 0x7fc9e4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:41.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.912+0000 7fca0259c640 1 --2- 172.21.15.163:0/259212813 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc9e40762d0 0x7fc9e4078790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fca040b03a0 tx=0x7fc9f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:41.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:41.916+0000 7fc9f2ffd640 1 -- 172.21.15.163:0/259212813 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fca040d0370 con 0x7fca041f6180 2024-05-08T06:37:42.007 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T06:37:42.009 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T06:37:42.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.099+0000 7fca08c83640 1 -- 172.21.15.163:0/259212813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fca0406a550 con 0x7fca041f6180 2024-05-08T06:37:42.787 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.788+0000 7fc9f2ffd640 1 -- 172.21.15.163:0/259212813 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) ==== 169+0+0 (secure 0 0 0) 0x7fc9f8019030 con 0x7fca041f6180 2024-05-08T06:37:42.787 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' removed 2024-05-08T06:37:42.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.790+0000 7fca08c83640 1 -- 172.21.15.163:0/259212813 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc9e40762d0 msgr2=0x7fc9e4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:42.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.790+0000 7fca08c83640 1 --2- 172.21.15.163:0/259212813 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc9e40762d0 0x7fc9e4078790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fca040b03a0 tx=0x7fc9f4009040 comp rx=0 tx=0).stop 2024-05-08T06:37:42.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.790+0000 7fca08c83640 1 -- 172.21.15.163:0/259212813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca041f6180 msgr2=0x7fca04070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:42.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.790+0000 7fca08c83640 1 --2- 172.21.15.163:0/259212813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca041f6180 0x7fca04070700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8011570 tx=0x7fc9f8011a30 comp rx=0 tx=0).stop 2024-05-08T06:37:42.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.790+0000 7fca08c83640 1 -- 172.21.15.163:0/259212813 shutdown_connections 2024-05-08T06:37:42.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.790+0000 7fca08c83640 1 --2- 172.21.15.163:0/259212813 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc9e40762d0 0x7fc9e4078790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:42.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.790+0000 7fca08c83640 1 --2- 172.21.15.163:0/259212813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca041f6600 0x7fca04070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:42.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.790+0000 7fca08c83640 1 --2- 172.21.15.163:0/259212813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca041f6180 0x7fca04070700 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:42.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.790+0000 7fca08c83640 1 --2- 172.21.15.163:0/259212813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ec005210 0x7fca04159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:42.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.790+0000 7fca08c83640 1 -- 172.21.15.163:0/259212813 >> 172.21.15.163:0/259212813 conn(0x7fca040739a0 msgr2=0x7fca041638c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:42.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.790+0000 7fca08c83640 1 -- 172.21.15.163:0/259212813 shutdown_connections 2024-05-08T06:37:42.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.790+0000 7fca08c83640 1 -- 172.21.15.163:0/259212813 wait complete. 2024-05-08T06:37:42.805 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T06:37:42.947 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.948+0000 7f86cf511640 1 Processor -- start 2024-05-08T06:37:42.947 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.948+0000 7f86cf511640 1 -- start start 2024-05-08T06:37:42.950 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cf511640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f86c81603f0 0x7f86c81607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:42.950 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cf511640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c80b5f10 0x7f86c815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:42.950 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cf511640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f86c815d3b0 0x7f86c80d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:42.950 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cf511640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f86c80b6370 con 0x7f86c81603f0 2024-05-08T06:37:42.950 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cf511640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f86c80b4e70 con 0x7f86c815d3b0 2024-05-08T06:37:42.950 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cf511640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f86c80b5150 con 0x7f86c80b5f10 2024-05-08T06:37:42.950 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cd2ad640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f86c81603f0 0x7f86c81607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:42.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cd2ad640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f86c81603f0 0x7f86c81607d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33856/0 (socket says 172.21.15.163:33856) 2024-05-08T06:37:42.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cd2ad640 1 -- 172.21.15.163:0/2852789486 learned_addr learned my addr 172.21.15.163:0/2852789486 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:42.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cdaae640 1 --2- 172.21.15.163:0/2852789486 >> v2:172.21.15.192:3300/0 conn(0x7f86c815d3b0 0x7f86c80d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:42.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86ccaac640 1 --2- 172.21.15.163:0/2852789486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c80b5f10 0x7f86c815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:42.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cd2ad640 1 -- 172.21.15.163:0/2852789486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c80b5f10 msgr2=0x7f86c815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:42.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cd2ad640 1 --2- 172.21.15.163:0/2852789486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c80b5f10 0x7f86c815ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:42.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cd2ad640 1 -- 172.21.15.163:0/2852789486 >> v2:172.21.15.192:3300/0 conn(0x7f86c815d3b0 msgr2=0x7f86c80d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:42.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cd2ad640 1 --2- 172.21.15.163:0/2852789486 >> v2:172.21.15.192:3300/0 conn(0x7f86c815d3b0 0x7f86c80d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:42.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cd2ad640 1 -- 172.21.15.163:0/2852789486 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c80b5430 con 0x7f86c81603f0 2024-05-08T06:37:42.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86cdaae640 1 --2- 172.21.15.163:0/2852789486 >> v2:172.21.15.192:3300/0 conn(0x7f86c815d3b0 0x7f86c80d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:42.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.949+0000 7f86ccaac640 1 --2- 172.21.15.163:0/2852789486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c80b5f10 0x7f86c815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:42.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.950+0000 7f86cd2ad640 1 --2- 172.21.15.163:0/2852789486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c81603f0 0x7f86c81607d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f86c80b46e0 tx=0x7f86c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10518c77f2745514 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:42.951 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.950+0000 7f86bffff640 1 -- 172.21.15.163:0/2852789486 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4037070 con 0x7f86c81603f0 2024-05-08T06:37:42.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.950+0000 7f86bffff640 1 -- 172.21.15.163:0/2852789486 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c4032440 con 0x7f86c81603f0 2024-05-08T06:37:42.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.950+0000 7f86bffff640 1 -- 172.21.15.163:0/2852789486 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4032810 con 0x7f86c81603f0 2024-05-08T06:37:42.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.951+0000 7f86cf511640 1 -- 172.21.15.163:0/2852789486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c81603f0 msgr2=0x7f86c81607d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:42.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.951+0000 7f86cf511640 1 --2- 172.21.15.163:0/2852789486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c81603f0 0x7f86c81607d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f86c80b46e0 tx=0x7f86c402a1e0 comp rx=0 tx=0).stop 2024-05-08T06:37:42.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.951+0000 7f86cf511640 1 -- 172.21.15.163:0/2852789486 shutdown_connections 2024-05-08T06:37:42.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.951+0000 7f86cf511640 1 --2- 172.21.15.163:0/2852789486 >> v2:172.21.15.192:3300/0 conn(0x7f86c815d3b0 0x7f86c80d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:42.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.951+0000 7f86cf511640 1 --2- 172.21.15.163:0/2852789486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c80b5f10 0x7f86c815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:42.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.951+0000 7f86cf511640 1 --2- 172.21.15.163:0/2852789486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c81603f0 0x7f86c81607d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:42.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.951+0000 7f86cf511640 1 -- 172.21.15.163:0/2852789486 >> 172.21.15.163:0/2852789486 conn(0x7f86c806c500 msgr2=0x7f86c806c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:42.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.951+0000 7f86cf511640 1 -- 172.21.15.163:0/2852789486 shutdown_connections 2024-05-08T06:37:42.952 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.951+0000 7f86cf511640 1 -- 172.21.15.163:0/2852789486 wait complete. 2024-05-08T06:37:42.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.952+0000 7f86cf511640 1 Processor -- start 2024-05-08T06:37:42.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.952+0000 7f86cf511640 1 -- start start 2024-05-08T06:37:42.954 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cf511640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c80b5f10 0x7f86c8072f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:42.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cf511640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c815d3b0 0x7f86c8143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:42.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cf511640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c81603f0 0x7f86c8156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:42.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cf511640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f86c80d17b0 con 0x7f86c81603f0 2024-05-08T06:37:42.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cf511640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f86c80b59f0 con 0x7f86c80b5f10 2024-05-08T06:37:42.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cf511640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f86c80d1930 con 0x7f86c815d3b0 2024-05-08T06:37:42.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cdaae640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c81603f0 0x7f86c8156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:42.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cdaae640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c81603f0 0x7f86c8156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33862/0 (socket says 172.21.15.163:33862) 2024-05-08T06:37:42.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cdaae640 1 -- 172.21.15.163:0/884708910 learned_addr learned my addr 172.21.15.163:0/884708910 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:42.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cd2ad640 1 --2- 172.21.15.163:0/884708910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c80b5f10 0x7f86c8072f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:42.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86ccaac640 1 --2- 172.21.15.163:0/884708910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c815d3b0 0x7f86c8143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:42.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cdaae640 1 -- 172.21.15.163:0/884708910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c815d3b0 msgr2=0x7f86c8143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:42.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cdaae640 1 --2- 172.21.15.163:0/884708910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c815d3b0 0x7f86c8143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:42.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cdaae640 1 -- 172.21.15.163:0/884708910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c80b5f10 msgr2=0x7f86c8072f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:42.955 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cdaae640 1 --2- 172.21.15.163:0/884708910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c80b5f10 0x7f86c8072f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:42.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cdaae640 1 -- 172.21.15.163:0/884708910 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c8144540 con 0x7f86c81603f0 2024-05-08T06:37:42.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.953+0000 7f86cdaae640 1 --2- 172.21.15.163:0/884708910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c81603f0 0x7f86c8156b60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f86c000ce70 tx=0x7f86c000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:42.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.954+0000 7f86bdffb640 1 -- 172.21.15.163:0/884708910 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0003b90 con 0x7f86c81603f0 2024-05-08T06:37:42.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.954+0000 7f86bdffb640 1 -- 172.21.15.163:0/884708910 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c0003d50 con 0x7f86c81603f0 2024-05-08T06:37:42.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.954+0000 7f86cf511640 1 -- 172.21.15.163:0/884708910 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c8144820 con 0x7f86c81603f0 2024-05-08T06:37:42.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.954+0000 7f86bdffb640 1 -- 172.21.15.163:0/884708910 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0004260 con 0x7f86c81603f0 2024-05-08T06:37:42.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.954+0000 7f86cf511640 1 -- 172.21.15.163:0/884708910 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c8144de0 con 0x7f86c81603f0 2024-05-08T06:37:42.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.955+0000 7f86cf511640 1 -- 172.21.15.163:0/884708910 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c806a550 con 0x7f86c81603f0 2024-05-08T06:37:42.956 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.955+0000 7f86bdffb640 1 -- 172.21.15.163:0/884708910 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f86c0004400 con 0x7f86c81603f0 2024-05-08T06:37:42.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.956+0000 7f86bdffb640 1 --2- 172.21.15.163:0/884708910 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f86ac0762d0 0x7f86ac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:42.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.956+0000 7f86cd2ad640 1 --2- 172.21.15.163:0/884708910 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f86ac0762d0 0x7f86ac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:42.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.956+0000 7f86bdffb640 1 -- 172.21.15.163:0/884708910 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f86c0097c80 con 0x7f86c81603f0 2024-05-08T06:37:42.957 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.956+0000 7f86cd2ad640 1 --2- 172.21.15.163:0/884708910 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f86ac0762d0 0x7f86ac078790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f86c4034000 tx=0x7f86c402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:42.959 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:42.960+0000 7f86bdffb640 1 -- 172.21.15.163:0/884708910 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f86c00611a0 con 0x7f86c81603f0 2024-05-08T06:37:43.145 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.146+0000 7f86cf511640 1 -- 172.21.15.163:0/884708910 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f86c80b4bb0 con 0x7f86c81603f0 2024-05-08T06:37:43.149 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.150+0000 7f86bdffb640 1 -- 172.21.15.163:0/884708910 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f86c00654a0 con 0x7f86c81603f0 2024-05-08T06:37:43.151 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.152+0000 7f86cf511640 1 -- 172.21.15.163:0/884708910 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f86ac0762d0 msgr2=0x7f86ac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:43.151 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.152+0000 7f86cf511640 1 --2- 172.21.15.163:0/884708910 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f86ac0762d0 0x7f86ac078790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f86c4034000 tx=0x7f86c402e000 comp rx=0 tx=0).stop 2024-05-08T06:37:43.151 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.152+0000 7f86cf511640 1 -- 172.21.15.163:0/884708910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c81603f0 msgr2=0x7f86c8156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:43.151 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.152+0000 7f86cf511640 1 --2- 172.21.15.163:0/884708910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c81603f0 0x7f86c8156b60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f86c000ce70 tx=0x7f86c000a710 comp rx=0 tx=0).stop 2024-05-08T06:37:43.151 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.152+0000 7f86cf511640 1 -- 172.21.15.163:0/884708910 shutdown_connections 2024-05-08T06:37:43.151 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.152+0000 7f86cf511640 1 --2- 172.21.15.163:0/884708910 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f86ac0762d0 0x7f86ac078790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:43.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.152+0000 7f86cf511640 1 --2- 172.21.15.163:0/884708910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c81603f0 0x7f86c8156b60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:43.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.152+0000 7f86cf511640 1 --2- 172.21.15.163:0/884708910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c815d3b0 0x7f86c8143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:43.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.152+0000 7f86cf511640 1 --2- 172.21.15.163:0/884708910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c80b5f10 0x7f86c8072f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:43.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.152+0000 7f86cf511640 1 -- 172.21.15.163:0/884708910 >> 172.21.15.163:0/884708910 conn(0x7f86c806c500 msgr2=0x7f86c815c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:43.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.152+0000 7f86cf511640 1 -- 172.21.15.163:0/884708910 shutdown_connections 2024-05-08T06:37:43.152 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.152+0000 7f86cf511640 1 -- 172.21.15.163:0/884708910 wait complete. 2024-05-08T06:37:43.162 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_flush.TestFlush.test_flush' 2024-05-08T06:37:43.262 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.263+0000 7fc319e45640 1 Processor -- start 2024-05-08T06:37:43.262 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.263+0000 7fc319e45640 1 -- start start 2024-05-08T06:37:43.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.264+0000 7fc319e45640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc314159a30 0x7fc3140b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.264+0000 7fc319e45640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc314159e10 0x7fc31406d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.264+0000 7fc319e45640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc31406dc90 con 0x7fc31406d870 2024-05-08T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.264+0000 7fc319e45640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fc31415ffc0 con 0x7fc314159e10 2024-05-08T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.264+0000 7fc319e45640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc3141602a0 con 0x7fc314159a30 2024-05-08T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.264+0000 7fc312ffd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc314159e10 0x7fc31406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.264+0000 7fc313fff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc31406d870 0x7fc3141631d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:41634/0 (socket says 172.21.15.163:41634) 2024-05-08T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.264+0000 7fc313fff640 1 -- 172.21.15.163:0/1865225156 learned_addr learned my addr 172.21.15.163:0/1865225156 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.264+0000 7fc312ffd640 1 -- 172.21.15.163:0/1865225156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc314159a30 msgr2=0x7fc3140b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.264+0000 7fc312ffd640 1 --2- 172.21.15.163:0/1865225156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc314159a30 0x7fc3140b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.264+0000 7fc312ffd640 1 -- 172.21.15.163:0/1865225156 >> v1:172.21.15.163:6789/0 conn(0x7fc31406d870 legacy=0x7fc3141631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.264+0000 7fc3127fc640 1 -- 172.21.15.163:0/1865225156 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2321573880 0 0) 0x7fc31406dc90 con 0x7fc31406d870 2024-05-08T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.264+0000 7fc312ffd640 1 -- 172.21.15.163:0/1865225156 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc314160580 con 0x7fc314159e10 2024-05-08T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.265+0000 7fc312ffd640 1 --2- 172.21.15.163:0/1865225156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc314159e10 0x7fc31406d330 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc3080097a0 tx=0x7fc30802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eba683ce53b1275 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:43.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.265+0000 7fc3127fc640 1 -- 172.21.15.163:0/1865225156 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308037070 con 0x7fc314159e10 2024-05-08T06:37:43.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.265+0000 7fc3127fc640 1 -- 172.21.15.163:0/1865225156 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc30802adb0 con 0x7fc314159e10 2024-05-08T06:37:43.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.265+0000 7fc3127fc640 1 -- 172.21.15.163:0/1865225156 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308032660 con 0x7fc314159e10 2024-05-08T06:37:43.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.266+0000 7fc319e45640 1 -- 172.21.15.163:0/1865225156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc314159e10 msgr2=0x7fc31406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:43.266 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.266+0000 7fc319e45640 1 --2- 172.21.15.163:0/1865225156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc314159e10 0x7fc31406d330 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc3080097a0 tx=0x7fc30802a060 comp rx=0 tx=0).stop 2024-05-08T06:37:43.266 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.266+0000 7fc319e45640 1 -- 172.21.15.163:0/1865225156 shutdown_connections 2024-05-08T06:37:43.266 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.266+0000 7fc319e45640 1 --2- 172.21.15.163:0/1865225156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc314159e10 0x7fc31406d330 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:43.266 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.266+0000 7fc319e45640 1 --2- 172.21.15.163:0/1865225156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc314159a30 0x7fc3140b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:43.266 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.266+0000 7fc319e45640 1 -- 172.21.15.163:0/1865225156 >> 172.21.15.163:0/1865225156 conn(0x7fc314072bd0 msgr2=0x7fc314072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:43.266 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.266+0000 7fc319e45640 1 -- 172.21.15.163:0/1865225156 shutdown_connections 2024-05-08T06:37:43.266 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.266+0000 7fc319e45640 1 -- 172.21.15.163:0/1865225156 wait complete. 2024-05-08T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.267+0000 7fc319e45640 1 Processor -- start 2024-05-08T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.267+0000 7fc319e45640 1 -- start start 2024-05-08T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc319e45640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc31406d870 0x7fc314150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc319e45640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc314159a30 0x7fc3141f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc319e45640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc314159e10 0x7fc3141f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc319e45640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc31415fd60 con 0x7fc314159a30 2024-05-08T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc319e45640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc31415fbe0 con 0x7fc314159e10 2024-05-08T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc319e45640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc3140b4ea0 con 0x7fc31406d870 2024-05-08T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc312ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc314159a30 0x7fc3141f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc312ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc314159a30 0x7fc3141f4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33870/0 (socket says 172.21.15.163:33870) 2024-05-08T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc312ffd640 1 -- 172.21.15.163:0/2978960548 learned_addr learned my addr 172.21.15.163:0/2978960548 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc3137fe640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc31406d870 0x7fc314150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc313fff640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc314159e10 0x7fc3141f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc312ffd640 1 -- 172.21.15.163:0/2978960548 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc31406d870 msgr2=0x7fc314150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc312ffd640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc31406d870 0x7fc314150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc312ffd640 1 -- 172.21.15.163:0/2978960548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc314159e10 msgr2=0x7fc3141f8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc312ffd640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc314159e10 0x7fc3141f8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc312ffd640 1 -- 172.21.15.163:0/2978960548 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc314161240 con 0x7fc314159a30 2024-05-08T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc313fff640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc314159e10 0x7fc3141f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc312ffd640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc314159a30 0x7fc3141f4dd0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc3080097a0 tx=0x7fc3080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc2fbfff640 1 -- 172.21.15.163:0/2978960548 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308042070 con 0x7fc314159a30 2024-05-08T06:37:43.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc2fbfff640 1 -- 172.21.15.163:0/2978960548 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc30803e670 con 0x7fc314159a30 2024-05-08T06:37:43.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc319e45640 1 -- 172.21.15.163:0/2978960548 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc314161520 con 0x7fc314159a30 2024-05-08T06:37:43.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc2fbfff640 1 -- 172.21.15.163:0/2978960548 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308037040 con 0x7fc314159a30 2024-05-08T06:37:43.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.268+0000 7fc319e45640 1 -- 172.21.15.163:0/2978960548 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3141f5310 con 0x7fc314159a30 2024-05-08T06:37:43.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.269+0000 7fc319e45640 1 -- 172.21.15.163:0/2978960548 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc31406a4b0 con 0x7fc314159a30 2024-05-08T06:37:43.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.270+0000 7fc2fbfff640 1 -- 172.21.15.163:0/2978960548 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc308045070 con 0x7fc314159a30 2024-05-08T06:37:43.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.271+0000 7fc2fbfff640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc300076300 0x7fc3000787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:43.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.271+0000 7fc2fbfff640 1 -- 172.21.15.163:0/2978960548 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fc3080b7bc0 con 0x7fc314159a30 2024-05-08T06:37:43.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.271+0000 7fc3137fe640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc300076300 0x7fc3000787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:43.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.271+0000 7fc3137fe640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc300076300 0x7fc3000787c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc30400b810 tx=0x7fc304008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:43.274 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.275+0000 7fc2fbfff640 1 -- 172.21.15.163:0/2978960548 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc3080817c0 con 0x7fc314159a30 2024-05-08T06:37:43.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.530+0000 7fc319e45640 1 -- 172.21.15.163:0/2978960548 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_flush.TestFlush.test_flush"]} v 0) -- 0x7fc3140b12b0 con 0x7fc314159a30 2024-05-08T06:37:43.788 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.789+0000 7fc2fbfff640 1 -- 172.21.15.163:0/2978960548 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_flush.TestFlush.test_flush"]}]=0 v82) ==== 123+0+0 (secure 0 0 0) 0x7fc3080810c0 con 0x7fc314159a30 2024-05-08T06:37:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.791+0000 7fc319e45640 1 -- 172.21.15.163:0/2978960548 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc300076300 msgr2=0x7fc3000787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.791+0000 7fc319e45640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc300076300 0x7fc3000787c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc30400b810 tx=0x7fc304008040 comp rx=0 tx=0).stop 2024-05-08T06:37:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.791+0000 7fc319e45640 1 -- 172.21.15.163:0/2978960548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc314159a30 msgr2=0x7fc3141f4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.791+0000 7fc319e45640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc314159a30 0x7fc3141f4dd0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc3080097a0 tx=0x7fc3080046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.791+0000 7fc319e45640 1 -- 172.21.15.163:0/2978960548 shutdown_connections 2024-05-08T06:37:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.791+0000 7fc319e45640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fc300076300 0x7fc3000787c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.791+0000 7fc319e45640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc314159e10 0x7fc3141f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.791+0000 7fc319e45640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc314159a30 0x7fc3141f4dd0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.791+0000 7fc319e45640 1 --2- 172.21.15.163:0/2978960548 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc31406d870 0x7fc314150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.791+0000 7fc319e45640 1 -- 172.21.15.163:0/2978960548 >> 172.21.15.163:0/2978960548 conn(0x7fc314072bd0 msgr2=0x7fc314159200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.791+0000 7fc319e45640 1 -- 172.21.15.163:0/2978960548 shutdown_connections 2024-05-08T06:37:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:43.792+0000 7fc319e45640 1 -- 172.21.15.163:0/2978960548 wait complete. 2024-05-08T06:37:43.838 INFO:tasks.cephfs_test_runner:test_flush (tasks.cephfs.test_flush.TestFlush) ... ERROR 2024-05-08T06:37:43.839 INFO:tasks.cephfs_test_runner: 2024-05-08T06:37:43.839 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-08T06:37:43.839 INFO:tasks.cephfs_test_runner:ERROR: test_flush (tasks.cephfs.test_flush.TestFlush) 2024-05-08T06:37:43.839 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-08T06:37:43.839 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-08T06:37:43.839 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_joscollin_ceph_6d0b29a4704de72eab52765a276866448dc1f6a5/qa/tasks/cephfs/test_flush.py", line 52, in test_flush 2024-05-08T06:37:43.839 INFO:tasks.cephfs_test_runner: self.assertEqual(summary_output, 2024-05-08T06:37:43.839 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/unittest/case.py", line 912, in assertEqual 2024-05-08T06:37:43.839 INFO:tasks.cephfs_test_runner: assertion_func(first, second, msg=msg) 2024-05-08T06:37:43.839 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/unittest/case.py", line 1292, in assertMultiLineEqual 2024-05-08T06:37:43.839 INFO:tasks.cephfs_test_runner: self.fail(self._formatMessage(msg, standardMsg)) 2024-05-08T06:37:43.839 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/unittest/case.py", line 753, in fail 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner: raise self.failureException(msg) 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner:AssertionError: 'Events by type:\n SUBTREEMAP: 1\n UPDATE: 1\nErrors: 0' != 'Events by type:\n SUBTREEMAP: 1\nErrors: 0' 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner: Events by type: 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner: SUBTREEMAP: 1 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner:- UPDATE: 1 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner: Errors: 0 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner: 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner: 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_joscollin_ceph_6d0b29a4704de72eab52765a276866448dc1f6a5/qa/tasks/cephfs/test_flush.py", line 78, in test_flush 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner: self.assertEqual(self.fs.journal_tool(["event", "get", "summary"], 0), 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_joscollin_ceph_6d0b29a4704de72eab52765a276866448dc1f6a5/qa/tasks/cephfs/filesystem.py", line 1731, in journal_tool 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner: return self._run_tool("cephfs-journal-tool", args, fs_rank, quiet) 2024-05-08T06:37:43.840 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_joscollin_ceph_6d0b29a4704de72eab52765a276866448dc1f6a5/qa/tasks/cephfs/filesystem.py", line 1710, in _run_tool 2024-05-08T06:37:43.841 INFO:tasks.cephfs_test_runner: r = self.tool_remote.sh(script=base_args + args, stdout=StringIO()).strip() 2024-05-08T06:37:43.841 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 97, in sh 2024-05-08T06:37:43.841 INFO:tasks.cephfs_test_runner: proc = self.run(**kwargs) 2024-05-08T06:37:43.841 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run 2024-05-08T06:37:43.841 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-05-08T06:37:43.841 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run 2024-05-08T06:37:43.841 INFO:tasks.cephfs_test_runner: r.wait() 2024-05-08T06:37:43.841 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait 2024-05-08T06:37:43.841 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-05-08T06:37:43.841 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-05-08T06:37:43.841 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-05-08T06:37:43.841 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi163 with status 255: 'cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event get summary' 2024-05-08T06:37:43.841 INFO:tasks.cephfs_test_runner: 2024-05-08T06:37:43.841 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-08T06:37:43.842 INFO:tasks.cephfs_test_runner:Ran 1 test in 38.477s 2024-05-08T06:37:43.842 INFO:tasks.cephfs_test_runner: 2024-05-08T06:37:43.842 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-05-08T06:37:43.842 INFO:tasks.cephfs_test_runner: 2024-05-08T06:37:43.842 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-08T06:37:43.842 INFO:tasks.cephfs_test_runner:ERROR: test_flush (tasks.cephfs.test_flush.TestFlush) 2024-05-08T06:37:43.842 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-08T06:37:43.842 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-08T06:37:43.842 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_joscollin_ceph_6d0b29a4704de72eab52765a276866448dc1f6a5/qa/tasks/cephfs/test_flush.py", line 52, in test_flush 2024-05-08T06:37:43.842 INFO:tasks.cephfs_test_runner: self.assertEqual(summary_output, 2024-05-08T06:37:43.842 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/unittest/case.py", line 912, in assertEqual 2024-05-08T06:37:43.842 INFO:tasks.cephfs_test_runner: assertion_func(first, second, msg=msg) 2024-05-08T06:37:43.843 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/unittest/case.py", line 1292, in assertMultiLineEqual 2024-05-08T06:37:43.843 INFO:tasks.cephfs_test_runner: self.fail(self._formatMessage(msg, standardMsg)) 2024-05-08T06:37:43.843 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/unittest/case.py", line 753, in fail 2024-05-08T06:37:43.843 INFO:tasks.cephfs_test_runner: raise self.failureException(msg) 2024-05-08T06:37:43.843 INFO:tasks.cephfs_test_runner:AssertionError: 'Events by type:\n SUBTREEMAP: 1\n UPDATE: 1\nErrors: 0' != 'Events by type:\n SUBTREEMAP: 1\nErrors: 0' 2024-05-08T06:37:43.843 INFO:tasks.cephfs_test_runner: Events by type: 2024-05-08T06:37:43.843 INFO:tasks.cephfs_test_runner: SUBTREEMAP: 1 2024-05-08T06:37:43.843 INFO:tasks.cephfs_test_runner:- UPDATE: 1 2024-05-08T06:37:43.843 INFO:tasks.cephfs_test_runner: Errors: 0 2024-05-08T06:37:43.843 INFO:tasks.cephfs_test_runner: 2024-05-08T06:37:43.843 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2024-05-08T06:37:43.843 INFO:tasks.cephfs_test_runner: 2024-05-08T06:37:43.843 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-08T06:37:43.844 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_joscollin_ceph_6d0b29a4704de72eab52765a276866448dc1f6a5/qa/tasks/cephfs/test_flush.py", line 78, in test_flush 2024-05-08T06:37:43.844 INFO:tasks.cephfs_test_runner: self.assertEqual(self.fs.journal_tool(["event", "get", "summary"], 0), 2024-05-08T06:37:43.844 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_joscollin_ceph_6d0b29a4704de72eab52765a276866448dc1f6a5/qa/tasks/cephfs/filesystem.py", line 1731, in journal_tool 2024-05-08T06:37:43.844 INFO:tasks.cephfs_test_runner: return self._run_tool("cephfs-journal-tool", args, fs_rank, quiet) 2024-05-08T06:37:43.844 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_joscollin_ceph_6d0b29a4704de72eab52765a276866448dc1f6a5/qa/tasks/cephfs/filesystem.py", line 1710, in _run_tool 2024-05-08T06:37:43.844 INFO:tasks.cephfs_test_runner: r = self.tool_remote.sh(script=base_args + args, stdout=StringIO()).strip() 2024-05-08T06:37:43.844 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 97, in sh 2024-05-08T06:37:43.844 INFO:tasks.cephfs_test_runner: proc = self.run(**kwargs) 2024-05-08T06:37:43.844 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run 2024-05-08T06:37:43.844 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-05-08T06:37:43.844 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run 2024-05-08T06:37:43.844 INFO:tasks.cephfs_test_runner: r.wait() 2024-05-08T06:37:43.844 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait 2024-05-08T06:37:43.844 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-05-08T06:37:43.845 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-05-08T06:37:43.845 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-05-08T06:37:43.845 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi163 with status 255: 'cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event get summary' 2024-05-08T06:37:43.845 INFO:tasks.cephfs_test_runner: 2024-05-08T06:37:43.845 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_joscollin_ceph_6d0b29a4704de72eab52765a276866448dc1f6a5/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_flush (tasks.cephfs.test_flush.TestFlush) 2024-05-08T06:37:44.125 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=ccf80d23be9d4f99af39ad8396cdb936 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_joscollin_ceph_6d0b29a4704de72eab52765a276866448dc1f6a5/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_flush (tasks.cephfs.test_flush.TestFlush) 2024-05-08T06:37:44.127 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-08T06:37:44.135 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-08T06:37:44.153 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-08T06:37:44.154 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:37:44.154 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:44.174 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18493 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T06:37:44.174 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T06:37:44.174 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:44.231 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18493 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T06:37:44.231 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:37:44.231 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:44.251 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056322,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497652k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12931 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497652k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T06:37:44.251 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T06:37:44.251 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:37:44.309 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056322,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497652k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12931 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497652k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T06:37:44.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:44.309 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-05-08T06:37:44.326 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T06:37:44.326 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T06:37:44.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:44.326 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T06:37:44.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:44.392 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T06:37:44.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:44.458 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2024-05-08T06:37:44.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:44.556 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-05-08T06:37:44.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:44.571 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2024-05-08T06:37:44.640 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2024-05-08T06:37:44.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:37:44.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:44.642 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-05-08T06:37:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-08T06:37:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T06:37:44.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:44.662 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T06:37:44.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:44.732 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T06:37:44.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:44.802 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2024-05-08T06:37:44.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:44.893 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-05-08T06:37:44.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:37:44.909 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2024-05-08T06:37:44.977 INFO:teuthology.orchestra.run.smithi192.stderr:Cannot find device "ceph-brx" 2024-05-08T06:37:44.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:37:44.979 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-08T06:37:44.993 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T06:37:44.993 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:37:45.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.096+0000 7f95d6332640 1 Processor -- start 2024-05-08T06:37:45.096 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.096+0000 7f95d6332640 1 -- start start 2024-05-08T06:37:45.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.097+0000 7f95d6332640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f95d00b54d0 0x7f95d00b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.097+0000 7f95d6332640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95d00b46e0 0x7f95d006f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.097+0000 7f95d6332640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d0072cb0 con 0x7f95d015b8a0 2024-05-08T06:37:45.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.097+0000 7f95d6332640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f95d00af910 con 0x7f95d00b54d0 2024-05-08T06:37:45.100 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.097+0000 7f95d6332640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f95d00b5140 con 0x7f95d00b46e0 2024-05-08T06:37:45.100 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.097+0000 7f95cf7fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f95d00b54d0 0x7f95d00b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.100 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.098+0000 7f95cffff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f95d015b8a0 0x7f95d00725a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:41650/0 (socket says 172.21.15.163:41650) 2024-05-08T06:37:45.100 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.098+0000 7f95cffff640 1 -- 172.21.15.163:0/755594650 learned_addr learned my addr 172.21.15.163:0/755594650 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:45.100 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.098+0000 7f95d48cf640 1 --2- 172.21.15.163:0/755594650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95d00b46e0 0x7f95d006f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.100 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.098+0000 7f95cf7fe640 1 -- 172.21.15.163:0/755594650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95d00b46e0 msgr2=0x7f95d006f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:45.100 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.098+0000 7f95cf7fe640 1 --2- 172.21.15.163:0/755594650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95d00b46e0 0x7f95d006f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.098+0000 7f95cf7fe640 1 -- 172.21.15.163:0/755594650 >> v1:172.21.15.163:6789/0 conn(0x7f95d015b8a0 legacy=0x7f95d00725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.098+0000 7f95cf7fe640 1 -- 172.21.15.163:0/755594650 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d00b4b80 con 0x7f95d00b54d0 2024-05-08T06:37:45.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.098+0000 7f95ceffd640 1 -- 172.21.15.163:0/755594650 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3718719233 0 0) 0x7f95d0072cb0 con 0x7f95d015b8a0 2024-05-08T06:37:45.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.098+0000 7f95d48cf640 1 --2- 172.21.15.163:0/755594650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95d00b46e0 0x7f95d006f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:45.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.099+0000 7f95cf7fe640 1 --2- 172.21.15.163:0/755594650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95d00b54d0 0x7f95d00b41a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f95c40097a0 tx=0x7f95c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2b285a24237d6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:45.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.099+0000 7f95ceffd640 1 -- 172.21.15.163:0/755594650 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4037070 con 0x7f95d00b54d0 2024-05-08T06:37:45.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.099+0000 7f95ceffd640 1 -- 172.21.15.163:0/755594650 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c4032440 con 0x7f95d00b54d0 2024-05-08T06:37:45.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.099+0000 7f95ceffd640 1 -- 172.21.15.163:0/755594650 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4032810 con 0x7f95d00b54d0 2024-05-08T06:37:45.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.099+0000 7f95d6332640 1 -- 172.21.15.163:0/755594650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95d00b54d0 msgr2=0x7f95d00b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.099+0000 7f95d6332640 1 --2- 172.21.15.163:0/755594650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95d00b54d0 0x7f95d00b41a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f95c40097a0 tx=0x7f95c402a180 comp rx=0 tx=0).stop 2024-05-08T06:37:45.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.099+0000 7f95d6332640 1 -- 172.21.15.163:0/755594650 shutdown_connections 2024-05-08T06:37:45.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.099+0000 7f95d6332640 1 --2- 172.21.15.163:0/755594650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95d00b46e0 0x7f95d006f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.099+0000 7f95d6332640 1 --2- 172.21.15.163:0/755594650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95d00b54d0 0x7f95d00b41a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.099+0000 7f95d6332640 1 -- 172.21.15.163:0/755594650 >> 172.21.15.163:0/755594650 conn(0x7f95d006c500 msgr2=0x7f95d006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:45.102 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.099+0000 7f95d6332640 1 -- 172.21.15.163:0/755594650 shutdown_connections 2024-05-08T06:37:45.103 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.099+0000 7f95d6332640 1 -- 172.21.15.163:0/755594650 wait complete. 2024-05-08T06:37:45.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.100+0000 7f95d6332640 1 Processor -- start 2024-05-08T06:37:45.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.101+0000 7f95d6332640 1 -- start start 2024-05-08T06:37:45.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.101+0000 7f95d6332640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95d00b46e0 0x7f95d01f1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.101+0000 7f95d6332640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95d00b54d0 0x7f95d0150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.101+0000 7f95d6332640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d015b8a0 0x7f95d0151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.101+0000 7f95d6332640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f95d006f5d0 con 0x7f95d00b54d0 2024-05-08T06:37:45.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.101+0000 7f95d6332640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f95d00af5a0 con 0x7f95d015b8a0 2024-05-08T06:37:45.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.101+0000 7f95d6332640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f95d006f750 con 0x7f95d00b46e0 2024-05-08T06:37:45.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.101+0000 7f95cf7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95d00b54d0 0x7f95d0150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95cf7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95d00b54d0 0x7f95d0150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33876/0 (socket says 172.21.15.163:33876) 2024-05-08T06:37:45.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95cf7fe640 1 -- 172.21.15.163:0/2971298866 learned_addr learned my addr 172.21.15.163:0/2971298866 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:45.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95cffff640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95d00b46e0 0x7f95d01f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95d48cf640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d015b8a0 0x7f95d0151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95cf7fe640 1 -- 172.21.15.163:0/2971298866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95d00b46e0 msgr2=0x7f95d01f1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95cf7fe640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95d00b46e0 0x7f95d01f1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95cf7fe640 1 -- 172.21.15.163:0/2971298866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d015b8a0 msgr2=0x7f95d0151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95cf7fe640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d015b8a0 0x7f95d0151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95cf7fe640 1 -- 172.21.15.163:0/2971298866 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d015d9e0 con 0x7f95d00b54d0 2024-05-08T06:37:45.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95d48cf640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d015b8a0 0x7f95d0151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:45.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95cf7fe640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95d00b54d0 0x7f95d0150df0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f95c4035040 tx=0x7f95c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:45.108 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95ccff9640 1 -- 172.21.15.163:0/2971298866 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4037040 con 0x7f95d00b54d0 2024-05-08T06:37:45.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95ccff9640 1 -- 172.21.15.163:0/2971298866 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c4030770 con 0x7f95d00b54d0 2024-05-08T06:37:45.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95ccff9640 1 -- 172.21.15.163:0/2971298866 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4030b80 con 0x7f95d00b54d0 2024-05-08T06:37:45.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95d6332640 1 -- 172.21.15.163:0/2971298866 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d015dcc0 con 0x7f95d00b54d0 2024-05-08T06:37:45.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.102+0000 7f95d6332640 1 -- 172.21.15.163:0/2971298866 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d015e280 con 0x7f95d00b54d0 2024-05-08T06:37:45.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.103+0000 7f95d6332640 1 -- 172.21.15.163:0/2971298866 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d0153130 con 0x7f95d00b54d0 2024-05-08T06:37:45.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.104+0000 7f95ccff9640 1 -- 172.21.15.163:0/2971298866 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f95c4044020 con 0x7f95d00b54d0 2024-05-08T06:37:45.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.104+0000 7f95ccff9640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f95bc0762d0 0x7f95bc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.104+0000 7f95cffff640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f95bc0762d0 0x7f95bc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.104+0000 7f95ccff9640 1 -- 172.21.15.163:0/2971298866 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f95c40b7860 con 0x7f95d00b54d0 2024-05-08T06:37:45.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.105+0000 7f95cffff640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f95bc0762d0 0x7f95bc078790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f95c0000f60 tx=0x7f95c0007b10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:45.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.108+0000 7f95ccff9640 1 -- 172.21.15.163:0/2971298866 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f95d0153130 con 0x7f95d00b54d0 2024-05-08T06:37:45.284 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.285+0000 7f95d6332640 1 -- 172.21.15.163:0/2971298866 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95d0156cf0 con 0x7f95bc0762d0 2024-05-08T06:37:45.285 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.286+0000 7f95ccff9640 1 -- 172.21.15.163:0/2971298866 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13389 (secure 0 0 0) 0x7f95d0156cf0 con 0x7f95bc0762d0 2024-05-08T06:37:45.285 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:45.286 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2024-05-08T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.288+0000 7f95d6332640 1 -- 172.21.15.163:0/2971298866 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f95bc0762d0 msgr2=0x7f95bc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.288+0000 7f95d6332640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f95bc0762d0 0x7f95bc078790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f95c0000f60 tx=0x7f95c0007b10 comp rx=0 tx=0).stop 2024-05-08T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.288+0000 7f95d6332640 1 -- 172.21.15.163:0/2971298866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95d00b54d0 msgr2=0x7f95d0150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.288+0000 7f95d6332640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95d00b54d0 0x7f95d0150df0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f95c4035040 tx=0x7f95c40046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.288+0000 7f95d6332640 1 -- 172.21.15.163:0/2971298866 shutdown_connections 2024-05-08T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.288+0000 7f95d6332640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f95bc0762d0 0x7f95bc078790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.288+0000 7f95d6332640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d015b8a0 0x7f95d0151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.288+0000 7f95d6332640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95d00b54d0 0x7f95d0150df0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.288+0000 7f95d6332640 1 --2- 172.21.15.163:0/2971298866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95d00b46e0 0x7f95d01f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.288+0000 7f95d6332640 1 -- 172.21.15.163:0/2971298866 >> 172.21.15.163:0/2971298866 conn(0x7f95d006c500 msgr2=0x7f95d00b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:45.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.288+0000 7f95d6332640 1 -- 172.21.15.163:0/2971298866 shutdown_connections 2024-05-08T06:37:45.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.288+0000 7f95d6332640 1 -- 172.21.15.163:0/2971298866 wait complete. 2024-05-08T06:37:45.299 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-05-08T06:37:44.109896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220200,"kb_used_data":4064,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754520,"statfs":{"total":773094113280,"available":772868628480,"internally_reserved":0,"allocated":4161536,"data_stored":2333720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14772,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-21,"num_read_kb":-21,"num_write":-75,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001008"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":132,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T06:37:31.616905+0000","last_change":"2024-05-08T06:36:31.710287+0000","last_active":"2024-05-08T06:37:31.616905+0000","last_peered":"2024-05-08T06:37:31.616905+0000","last_clean":"2024-05-08T06:37:31.616905+0000","last_became_active":"2024-05-08T06:36:31.709270+0000","last_became_peered":"2024-05-08T06:36:31.709270+0000","last_unstale":"2024-05-08T06:37:31.616905+0000","last_undegraded":"2024-05-08T06:37:31.616905+0000","last_fullsized":"2024-05-08T06:37:31.616905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_clean_scrub_stamp":"2024-05-08T06:36:30.122918+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:42.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":730665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":368640,"data_stored":140823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":368640,"data_stored":140879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":368640,"data_stored":140311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":389120,"data_stored":155095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":140277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":389120,"data_stored":155095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":954368,"data_stored":730575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:37:45.299 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:37:45.399 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.400+0000 7f8a927af640 1 Processor -- start 2024-05-08T06:37:45.399 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.400+0000 7f8a927af640 1 -- start start 2024-05-08T06:37:45.400 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.400+0000 7f8a927af640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a8c15f760 0x7f8a8c0b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.400+0000 7f8a927af640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8a8c0cfc10 0x7f8a8c0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.400+0000 7f8a927af640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a8c0d0070 con 0x7f8a8c0d0830 2024-05-08T06:37:45.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.401+0000 7f8a927af640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8a8c15fb40 con 0x7f8a8c0cfc10 2024-05-08T06:37:45.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.401+0000 7f8a927af640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8a8c0d36f0 con 0x7f8a8c15f760 2024-05-08T06:37:45.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.401+0000 7f8a8b7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8a8c0cfc10 0x7f8a8c0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.401+0000 7f8a8bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a8c15f760 0x7f8a8c0b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.401+0000 7f8a8b7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8a8c0cfc10 0x7f8a8c0d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:41612/0 (socket says 172.21.15.163:41612) 2024-05-08T06:37:45.401 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.401+0000 7f8a8b7fe640 1 -- 172.21.15.163:0/140720421 learned_addr learned my addr 172.21.15.163:0/140720421 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:45.402 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.401+0000 7f8a90d4c640 1 --1- 172.21.15.163:0/140720421 >> v1:172.21.15.192:6789/0 conn(0x7f8a8c0d0830 0x7f8a8c0b6d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.163:34460/0 (socket says 172.21.15.163:34460) 2024-05-08T06:37:45.402 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.401+0000 7f8a8bfff640 1 -- 172.21.15.163:0/140720421 >> v2:172.21.15.192:3300/0 conn(0x7f8a8c0cfc10 msgr2=0x7f8a8c0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:45.402 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.401+0000 7f8a8bfff640 1 --2- 172.21.15.163:0/140720421 >> v2:172.21.15.192:3300/0 conn(0x7f8a8c0cfc10 0x7f8a8c0d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.402 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.401+0000 7f8a8bfff640 1 -- 172.21.15.163:0/140720421 >> v1:172.21.15.192:6789/0 conn(0x7f8a8c0d0830 legacy=0x7f8a8c0b6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.402 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.401+0000 7f8a8bfff640 1 -- 172.21.15.163:0/140720421 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8c0d39d0 con 0x7f8a8c15f760 2024-05-08T06:37:45.402 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.401+0000 7f8a8b7fe640 1 --2- 172.21.15.163:0/140720421 >> v2:172.21.15.192:3300/0 conn(0x7f8a8c0cfc10 0x7f8a8c0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:45.405 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.402+0000 7f8a8bfff640 1 --2- 172.21.15.163:0/140720421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a8c15f760 0x7f8a8c0b4770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0097a0 tx=0x7f8a7c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a6a7ed361edf37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:45.405 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.402+0000 7f8a8affd640 1 -- 172.21.15.163:0/140720421 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c037070 con 0x7f8a8c15f760 2024-05-08T06:37:45.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.402+0000 7f8a8affd640 1 -- 172.21.15.163:0/140720421 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a7c02ae50 con 0x7f8a8c15f760 2024-05-08T06:37:45.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.402+0000 7f8a8affd640 1 -- 172.21.15.163:0/140720421 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c032660 con 0x7f8a8c15f760 2024-05-08T06:37:45.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.403+0000 7f8a927af640 1 -- 172.21.15.163:0/140720421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a8c15f760 msgr2=0x7f8a8c0b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.403+0000 7f8a927af640 1 --2- 172.21.15.163:0/140720421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a8c15f760 0x7f8a8c0b4770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0097a0 tx=0x7f8a7c02a100 comp rx=0 tx=0).stop 2024-05-08T06:37:45.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.403+0000 7f8a927af640 1 -- 172.21.15.163:0/140720421 shutdown_connections 2024-05-08T06:37:45.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.403+0000 7f8a927af640 1 --2- 172.21.15.163:0/140720421 >> v2:172.21.15.192:3300/0 conn(0x7f8a8c0cfc10 0x7f8a8c0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.406 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.403+0000 7f8a927af640 1 --2- 172.21.15.163:0/140720421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a8c15f760 0x7f8a8c0b4770 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.403+0000 7f8a927af640 1 -- 172.21.15.163:0/140720421 >> 172.21.15.163:0/140720421 conn(0x7f8a8c0740e0 msgr2=0x7f8a8c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:45.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.403+0000 7f8a927af640 1 -- 172.21.15.163:0/140720421 shutdown_connections 2024-05-08T06:37:45.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.403+0000 7f8a927af640 1 -- 172.21.15.163:0/140720421 wait complete. 2024-05-08T06:37:45.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.404+0000 7f8a927af640 1 Processor -- start 2024-05-08T06:37:45.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.404+0000 7f8a927af640 1 -- start start 2024-05-08T06:37:45.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a927af640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a8c0cfc10 0x7f8a8c1f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a927af640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a8c0d0830 0x7f8a8c154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.407 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a927af640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8c15f760 0x7f8a8c157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.408 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a927af640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c0d4270 con 0x7f8a8c15f760 2024-05-08T06:37:45.408 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a927af640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c0d3f90 con 0x7f8a8c0cfc10 2024-05-08T06:37:45.408 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a927af640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8a8c0b9720 con 0x7f8a8c0d0830 2024-05-08T06:37:45.408 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a90d4c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8c15f760 0x7f8a8c157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.408 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a90d4c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8c15f760 0x7f8a8c157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33880/0 (socket says 172.21.15.163:33880) 2024-05-08T06:37:45.408 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a90d4c640 1 -- 172.21.15.163:0/1155163834 learned_addr learned my addr 172.21.15.163:0/1155163834 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:45.408 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a8bfff640 1 --2- 172.21.15.163:0/1155163834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a8c0cfc10 0x7f8a8c1f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.408 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a8b7fe640 1 --2- 172.21.15.163:0/1155163834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a8c0d0830 0x7f8a8c154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.409 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a90d4c640 1 -- 172.21.15.163:0/1155163834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a8c0d0830 msgr2=0x7f8a8c154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.409 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a90d4c640 1 --2- 172.21.15.163:0/1155163834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a8c0d0830 0x7f8a8c154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.409 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a90d4c640 1 -- 172.21.15.163:0/1155163834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a8c0cfc10 msgr2=0x7f8a8c1f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.409 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a90d4c640 1 --2- 172.21.15.163:0/1155163834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a8c0cfc10 0x7f8a8c1f98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.409 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.405+0000 7f8a90d4c640 1 -- 172.21.15.163:0/1155163834 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8c1f9df0 con 0x7f8a8c15f760 2024-05-08T06:37:45.409 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.406+0000 7f8a90d4c640 1 --2- 172.21.15.163:0/1155163834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8c15f760 0x7f8a8c157470 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400c730 tx=0x7f8a8400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:45.409 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.406+0000 7f8a88ff9640 1 -- 172.21.15.163:0/1155163834 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84017070 con 0x7f8a8c15f760 2024-05-08T06:37:45.409 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.406+0000 7f8a88ff9640 1 -- 172.21.15.163:0/1155163834 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a8400ce40 con 0x7f8a8c15f760 2024-05-08T06:37:45.410 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.406+0000 7f8a927af640 1 -- 172.21.15.163:0/1155163834 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8c1549c0 con 0x7f8a8c15f760 2024-05-08T06:37:45.410 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.406+0000 7f8a88ff9640 1 -- 172.21.15.163:0/1155163834 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8400a840 con 0x7f8a8c15f760 2024-05-08T06:37:45.410 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.406+0000 7f8a927af640 1 -- 172.21.15.163:0/1155163834 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a8c156990 con 0x7f8a8c15f760 2024-05-08T06:37:45.411 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.407+0000 7f8a927af640 1 -- 172.21.15.163:0/1155163834 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a8c06a550 con 0x7f8a8c15f760 2024-05-08T06:37:45.411 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.407+0000 7f8a88ff9640 1 -- 172.21.15.163:0/1155163834 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8a8401a070 con 0x7f8a8c15f760 2024-05-08T06:37:45.411 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.408+0000 7f8a88ff9640 1 --2- 172.21.15.163:0/1155163834 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8a780762d0 0x7f8a78078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.411 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.408+0000 7f8a88ff9640 1 -- 172.21.15.163:0/1155163834 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f8a84096830 con 0x7f8a8c15f760 2024-05-08T06:37:45.412 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.408+0000 7f8a8bfff640 1 --2- 172.21.15.163:0/1155163834 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8a780762d0 0x7f8a78078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.412 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.408+0000 7f8a8bfff640 1 --2- 172.21.15.163:0/1155163834 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8a780762d0 0x7f8a78078790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c034000 tx=0x7f8a7c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:45.412 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.412+0000 7f8a88ff9640 1 -- 172.21.15.163:0/1155163834 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8a8c1f9df0 con 0x7f8a8c15f760 2024-05-08T06:37:45.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.587+0000 7f8a927af640 1 -- 172.21.15.163:0/1155163834 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a8c0cf0e0 con 0x7f8a780762d0 2024-05-08T06:37:45.587 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.588+0000 7f8a88ff9640 1 -- 172.21.15.163:0/1155163834 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13389 (secure 0 0 0) 0x7f8a8c0cf0e0 con 0x7f8a780762d0 2024-05-08T06:37:45.587 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:45.588 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2024-05-08T06:37:45.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.590+0000 7f8a927af640 1 -- 172.21.15.163:0/1155163834 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8a780762d0 msgr2=0x7f8a78078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.590+0000 7f8a927af640 1 --2- 172.21.15.163:0/1155163834 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8a780762d0 0x7f8a78078790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c034000 tx=0x7f8a7c02e000 comp rx=0 tx=0).stop 2024-05-08T06:37:45.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.590+0000 7f8a927af640 1 -- 172.21.15.163:0/1155163834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8c15f760 msgr2=0x7f8a8c157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.590+0000 7f8a927af640 1 --2- 172.21.15.163:0/1155163834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8c15f760 0x7f8a8c157470 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400c730 tx=0x7f8a8400cbf0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.590+0000 7f8a927af640 1 -- 172.21.15.163:0/1155163834 shutdown_connections 2024-05-08T06:37:45.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.590+0000 7f8a927af640 1 --2- 172.21.15.163:0/1155163834 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8a780762d0 0x7f8a78078790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.590+0000 7f8a927af640 1 --2- 172.21.15.163:0/1155163834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8c15f760 0x7f8a8c157470 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.590+0000 7f8a927af640 1 --2- 172.21.15.163:0/1155163834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a8c0d0830 0x7f8a8c154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.591 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.590+0000 7f8a927af640 1 --2- 172.21.15.163:0/1155163834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a8c0cfc10 0x7f8a8c1f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.591 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.590+0000 7f8a927af640 1 -- 172.21.15.163:0/1155163834 >> 172.21.15.163:0/1155163834 conn(0x7f8a8c0740e0 msgr2=0x7f8a8c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:45.591 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.590+0000 7f8a927af640 1 -- 172.21.15.163:0/1155163834 shutdown_connections 2024-05-08T06:37:45.591 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.590+0000 7f8a927af640 1 -- 172.21.15.163:0/1155163834 wait complete. 2024-05-08T06:37:45.600 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-05-08T06:37:44.109896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220200,"kb_used_data":4064,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754520,"statfs":{"total":773094113280,"available":772868628480,"internally_reserved":0,"allocated":4161536,"data_stored":2333720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14772,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-21,"num_read_kb":-21,"num_write":-75,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001008"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":132,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T06:37:31.616905+0000","last_change":"2024-05-08T06:36:31.710287+0000","last_active":"2024-05-08T06:37:31.616905+0000","last_peered":"2024-05-08T06:37:31.616905+0000","last_clean":"2024-05-08T06:37:31.616905+0000","last_became_active":"2024-05-08T06:36:31.709270+0000","last_became_peered":"2024-05-08T06:36:31.709270+0000","last_unstale":"2024-05-08T06:37:31.616905+0000","last_undegraded":"2024-05-08T06:37:31.616905+0000","last_fullsized":"2024-05-08T06:37:31.616905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_clean_scrub_stamp":"2024-05-08T06:36:30.122918+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:42.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":730665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":368640,"data_stored":140823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":368640,"data_stored":140879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":368640,"data_stored":140311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":389120,"data_stored":155095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":140277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":389120,"data_stored":155095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":954368,"data_stored":730575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:37:45.601 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T06:37:45.601 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:37:45.701 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.702+0000 7f2256f9d640 1 Processor -- start 2024-05-08T06:37:45.701 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.702+0000 7f2256f9d640 1 -- start start 2024-05-08T06:37:45.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.703+0000 7f2256f9d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f22500b54d0 0x7f22500b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.703+0000 7f2256f9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22500b46e0 0x7f225006f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.703+0000 7f2256f9d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2250072cb0 con 0x7f225015b8a0 2024-05-08T06:37:45.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.703+0000 7f2256f9d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f22500af910 con 0x7f22500b54d0 2024-05-08T06:37:45.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.703+0000 7f2256f9d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f22500b5140 con 0x7f22500b46e0 2024-05-08T06:37:45.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.703+0000 7f2247fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f22500b54d0 0x7f22500b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.703+0000 7f2247fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f22500b54d0 0x7f22500b41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33890/0 (socket says 172.21.15.163:33890) 2024-05-08T06:37:45.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.703+0000 7f2247fff640 1 -- 172.21.15.163:0/242759304 learned_addr learned my addr 172.21.15.163:0/242759304 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:45.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.703+0000 7f2247fff640 1 -- 172.21.15.163:0/242759304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22500b46e0 msgr2=0x7f225006f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:45.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.703+0000 7f225553a640 1 --2- 172.21.15.163:0/242759304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22500b46e0 0x7f225006f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.703+0000 7f2247fff640 1 --2- 172.21.15.163:0/242759304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22500b46e0 0x7f225006f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.703+0000 7f2247fff640 1 -- 172.21.15.163:0/242759304 >> v1:172.21.15.192:6789/0 conn(0x7f225015b8a0 legacy=0x7f22500725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:45.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.703+0000 7f2247fff640 1 -- 172.21.15.163:0/242759304 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22500b4b80 con 0x7f22500b54d0 2024-05-08T06:37:45.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.703+0000 7f225553a640 1 --2- 172.21.15.163:0/242759304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22500b46e0 0x7f225006f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:45.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.704+0000 7f2247fff640 1 --2- 172.21.15.163:0/242759304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22500b54d0 0x7f22500b41a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2240009870 tx=0x7f224002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e0e2ce2714c7e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:45.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.704+0000 7f22477fe640 1 -- 172.21.15.163:0/242759304 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240037070 con 0x7f22500b54d0 2024-05-08T06:37:45.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.704+0000 7f22477fe640 1 -- 172.21.15.163:0/242759304 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2240032440 con 0x7f22500b54d0 2024-05-08T06:37:45.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.704+0000 7f22477fe640 1 -- 172.21.15.163:0/242759304 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240032810 con 0x7f22500b54d0 2024-05-08T06:37:45.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.705+0000 7f2256f9d640 1 -- 172.21.15.163:0/242759304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22500b54d0 msgr2=0x7f22500b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.705+0000 7f2256f9d640 1 --2- 172.21.15.163:0/242759304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22500b54d0 0x7f22500b41a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2240009870 tx=0x7f224002a1c0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.705+0000 7f2256f9d640 1 -- 172.21.15.163:0/242759304 shutdown_connections 2024-05-08T06:37:45.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.705+0000 7f2256f9d640 1 --2- 172.21.15.163:0/242759304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22500b46e0 0x7f225006f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.705+0000 7f2256f9d640 1 --2- 172.21.15.163:0/242759304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22500b54d0 0x7f22500b41a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.705+0000 7f2256f9d640 1 -- 172.21.15.163:0/242759304 >> 172.21.15.163:0/242759304 conn(0x7f225006c500 msgr2=0x7f225006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:45.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.705+0000 7f2256f9d640 1 -- 172.21.15.163:0/242759304 shutdown_connections 2024-05-08T06:37:45.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.705+0000 7f2256f9d640 1 -- 172.21.15.163:0/242759304 wait complete. 2024-05-08T06:37:45.710 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.706+0000 7f2256f9d640 1 Processor -- start 2024-05-08T06:37:45.710 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.706+0000 7f2256f9d640 1 -- start start 2024-05-08T06:37:45.710 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2256f9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22500b46e0 0x7f225014bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.710 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2256f9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22500b54d0 0x7f2250097a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2256f9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f225015b8a0 0x7f225014f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2256f9d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f225006f5d0 con 0x7f22500b54d0 2024-05-08T06:37:45.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2256f9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f22500af5a0 con 0x7f225015b8a0 2024-05-08T06:37:45.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2256f9d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f225006f750 con 0x7f22500b46e0 2024-05-08T06:37:45.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2247fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22500b54d0 0x7f2250097a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2247fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22500b54d0 0x7f2250097a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33894/0 (socket says 172.21.15.163:33894) 2024-05-08T06:37:45.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2247fff640 1 -- 172.21.15.163:0/2870617443 learned_addr learned my addr 172.21.15.163:0/2870617443 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:45.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2254d39640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22500b46e0 0x7f225014bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f225553a640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f225015b8a0 0x7f225014f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2247fff640 1 -- 172.21.15.163:0/2870617443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22500b46e0 msgr2=0x7f225014bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2247fff640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22500b46e0 0x7f225014bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2247fff640 1 -- 172.21.15.163:0/2870617443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f225015b8a0 msgr2=0x7f225014f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2247fff640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f225015b8a0 0x7f225014f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2247fff640 1 -- 172.21.15.163:0/2870617443 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225014b140 con 0x7f22500b54d0 2024-05-08T06:37:45.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2254d39640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22500b46e0 0x7f225014bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:45.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f225553a640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f225015b8a0 0x7f225014f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:45.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.707+0000 7f2247fff640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22500b54d0 0x7f2250097a50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2240000c00 tx=0x7f22400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:45.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.708+0000 7f22457fa640 1 -- 172.21.15.163:0/2870617443 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240037040 con 0x7f22500b54d0 2024-05-08T06:37:45.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.708+0000 7f22457fa640 1 -- 172.21.15.163:0/2870617443 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2240042070 con 0x7f22500b54d0 2024-05-08T06:37:45.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.708+0000 7f2256f9d640 1 -- 172.21.15.163:0/2870617443 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22501475f0 con 0x7f22500b54d0 2024-05-08T06:37:45.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.708+0000 7f22457fa640 1 -- 172.21.15.163:0/2870617443 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240030bb0 con 0x7f22500b54d0 2024-05-08T06:37:45.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.708+0000 7f2256f9d640 1 -- 172.21.15.163:0/2870617443 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225014b6e0 con 0x7f22500b54d0 2024-05-08T06:37:45.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.709+0000 7f2256f9d640 1 -- 172.21.15.163:0/2870617443 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225006a550 con 0x7f22500b54d0 2024-05-08T06:37:45.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.709+0000 7f22457fa640 1 -- 172.21.15.163:0/2870617443 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f224002ad10 con 0x7f22500b54d0 2024-05-08T06:37:45.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.710+0000 7f22457fa640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f223c076300 0x7f223c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:45.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.710+0000 7f2254d39640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f223c076300 0x7f223c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:45.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.710+0000 7f22457fa640 1 -- 172.21.15.163:0/2870617443 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f22400b7d30 con 0x7f22500b54d0 2024-05-08T06:37:45.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.710+0000 7f2254d39640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f223c076300 0x7f223c0787c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f225006ce70 tx=0x7f224c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:45.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.714+0000 7f22457fa640 1 -- 172.21.15.163:0/2870617443 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f22400818b0 con 0x7f22500b54d0 2024-05-08T06:37:45.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.896+0000 7f2256f9d640 1 -- 172.21.15.163:0/2870617443 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f225006c980 con 0x7f223c076300 2024-05-08T06:37:45.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.897+0000 7f22457fa640 1 -- 172.21.15.163:0/2870617443 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13389 (secure 0 0 0) 0x7f225006c980 con 0x7f223c076300 2024-05-08T06:37:45.896 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:45.897 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2024-05-08T06:37:45.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.899+0000 7f2256f9d640 1 -- 172.21.15.163:0/2870617443 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f223c076300 msgr2=0x7f223c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.899+0000 7f2256f9d640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f223c076300 0x7f223c0787c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f225006ce70 tx=0x7f224c009040 comp rx=0 tx=0).stop 2024-05-08T06:37:45.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.899+0000 7f2256f9d640 1 -- 172.21.15.163:0/2870617443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22500b54d0 msgr2=0x7f2250097a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:45.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.899+0000 7f2256f9d640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22500b54d0 0x7f2250097a50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2240000c00 tx=0x7f22400046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.899+0000 7f2256f9d640 1 -- 172.21.15.163:0/2870617443 shutdown_connections 2024-05-08T06:37:45.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.899+0000 7f2256f9d640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f223c076300 0x7f223c0787c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.899+0000 7f2256f9d640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f225015b8a0 0x7f225014f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.899+0000 7f2256f9d640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22500b54d0 0x7f2250097a50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.899+0000 7f2256f9d640 1 --2- 172.21.15.163:0/2870617443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22500b46e0 0x7f225014bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:45.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.899+0000 7f2256f9d640 1 -- 172.21.15.163:0/2870617443 >> 172.21.15.163:0/2870617443 conn(0x7f225006c500 msgr2=0x7f2250153630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:45.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.899+0000 7f2256f9d640 1 -- 172.21.15.163:0/2870617443 shutdown_connections 2024-05-08T06:37:45.900 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:45.899+0000 7f2256f9d640 1 -- 172.21.15.163:0/2870617443 wait complete. 2024-05-08T06:37:45.909 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-05-08T06:37:44.109896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220200,"kb_used_data":4064,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754520,"statfs":{"total":773094113280,"available":772868628480,"internally_reserved":0,"allocated":4161536,"data_stored":2333720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14772,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-21,"num_read_kb":-21,"num_write":-75,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001008"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":132,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-08T06:37:31.616905+0000","last_change":"2024-05-08T06:36:31.710287+0000","last_active":"2024-05-08T06:37:31.616905+0000","last_peered":"2024-05-08T06:37:31.616905+0000","last_clean":"2024-05-08T06:37:31.616905+0000","last_became_active":"2024-05-08T06:36:31.709270+0000","last_became_peered":"2024-05-08T06:36:31.709270+0000","last_unstale":"2024-05-08T06:37:31.616905+0000","last_undegraded":"2024-05-08T06:37:31.616905+0000","last_fullsized":"2024-05-08T06:37:31.616905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_clean_scrub_stamp":"2024-05-08T06:36:30.122918+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:42.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":730665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":368640,"data_stored":140823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":368640,"data_stored":140879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":368640,"data_stored":140311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":389120,"data_stored":155095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":140277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":389120,"data_stored":155095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":954368,"data_stored":730575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:37:45.910 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:37:46.010 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.011+0000 7fbcc7504640 1 Processor -- start 2024-05-08T06:37:46.010 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.011+0000 7fbcc7504640 1 -- start start 2024-05-08T06:37:46.012 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.012+0000 7fbcc7504640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbcc0140070 0x7fbcc0071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:46.012 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.012+0000 7fbcc7504640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbcc00722f0 0x7fbcc0142810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:46.013 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.012+0000 7fbcc7504640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcc0142d50 0x7fbcc0160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:46.013 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.012+0000 7fbcc7504640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc01431a0 con 0x7fbcc0140070 2024-05-08T06:37:46.013 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.012+0000 7fbcc7504640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc0072e50 con 0x7fbcc00722f0 2024-05-08T06:37:46.013 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.012+0000 7fbcc7504640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc0140450 con 0x7fbcc0142d50 2024-05-08T06:37:46.013 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.012+0000 7fbcc52a0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbcc0140070 0x7fbcc0071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:46.013 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.012+0000 7fbcc52a0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbcc0140070 0x7fbcc0071db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33902/0 (socket says 172.21.15.163:33902) 2024-05-08T06:37:46.013 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.012+0000 7fbcc52a0640 1 -- 172.21.15.163:0/3610157183 learned_addr learned my addr 172.21.15.163:0/3610157183 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:46.013 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.013+0000 7fbcc5aa1640 1 --2- 172.21.15.163:0/3610157183 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcc0142d50 0x7fbcc0160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:46.014 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.013+0000 7fbcc4a9f640 1 --2- 172.21.15.163:0/3610157183 >> v2:172.21.15.192:3300/0 conn(0x7fbcc00722f0 0x7fbcc0142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:46.014 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.013+0000 7fbcc52a0640 1 -- 172.21.15.163:0/3610157183 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcc0142d50 msgr2=0x7fbcc0160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:46.014 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.013+0000 7fbcc52a0640 1 --2- 172.21.15.163:0/3610157183 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcc0142d50 0x7fbcc0160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:46.014 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.013+0000 7fbcc52a0640 1 -- 172.21.15.163:0/3610157183 >> v2:172.21.15.192:3300/0 conn(0x7fbcc00722f0 msgr2=0x7fbcc0142810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:46.014 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.013+0000 7fbcc52a0640 1 --2- 172.21.15.163:0/3610157183 >> v2:172.21.15.192:3300/0 conn(0x7fbcc00722f0 0x7fbcc0142810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:46.014 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.013+0000 7fbcc52a0640 1 -- 172.21.15.163:0/3610157183 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc0072890 con 0x7fbcc0140070 2024-05-08T06:37:46.014 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.013+0000 7fbcc52a0640 1 --2- 172.21.15.163:0/3610157183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcc0140070 0x7fbcc0071db0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0161d80 tx=0x7fbcbc02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935e83ed76d70ddc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:46.015 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.013+0000 7fbcb7fff640 1 -- 172.21.15.163:0/3610157183 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc037070 con 0x7fbcc0140070 2024-05-08T06:37:46.015 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.014+0000 7fbcb7fff640 1 -- 172.21.15.163:0/3610157183 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcbc02ae00 con 0x7fbcc0140070 2024-05-08T06:37:46.015 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.014+0000 7fbcb7fff640 1 -- 172.21.15.163:0/3610157183 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc032660 con 0x7fbcc0140070 2024-05-08T06:37:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.014+0000 7fbcc7504640 1 -- 172.21.15.163:0/3610157183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcc0140070 msgr2=0x7fbcc0071db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.014+0000 7fbcc7504640 1 --2- 172.21.15.163:0/3610157183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcc0140070 0x7fbcc0071db0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0161d80 tx=0x7fbcbc02a0b0 comp rx=0 tx=0).stop 2024-05-08T06:37:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.014+0000 7fbcc7504640 1 -- 172.21.15.163:0/3610157183 shutdown_connections 2024-05-08T06:37:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.014+0000 7fbcc7504640 1 --2- 172.21.15.163:0/3610157183 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcc0142d50 0x7fbcc0160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.014+0000 7fbcc7504640 1 --2- 172.21.15.163:0/3610157183 >> v2:172.21.15.192:3300/0 conn(0x7fbcc00722f0 0x7fbcc0142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.014+0000 7fbcc7504640 1 --2- 172.21.15.163:0/3610157183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcc0140070 0x7fbcc0071db0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:46.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.014+0000 7fbcc7504640 1 -- 172.21.15.163:0/3610157183 >> 172.21.15.163:0/3610157183 conn(0x7fbcc00b4da0 msgr2=0x7fbcc00b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:46.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.014+0000 7fbcc7504640 1 -- 172.21.15.163:0/3610157183 shutdown_connections 2024-05-08T06:37:46.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.014+0000 7fbcc7504640 1 -- 172.21.15.163:0/3610157183 wait complete. 2024-05-08T06:37:46.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.015+0000 7fbcc7504640 1 Processor -- start 2024-05-08T06:37:46.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.015+0000 7fbcc7504640 1 -- start start 2024-05-08T06:37:46.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc7504640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcc00722f0 0x7fbcc01f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:46.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc7504640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc0140070 0x7fbcc006de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:46.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc7504640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcc0142d50 0x7fbcc015bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:46.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc7504640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01435b0 con 0x7fbcc00722f0 2024-05-08T06:37:46.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc7504640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01408b0 con 0x7fbcc0140070 2024-05-08T06:37:46.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc7504640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc0161220 con 0x7fbcc0142d50 2024-05-08T06:37:46.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc52a0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcc00722f0 0x7fbcc01f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:46.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc52a0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcc00722f0 0x7fbcc01f8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33910/0 (socket says 172.21.15.163:33910) 2024-05-08T06:37:46.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc52a0640 1 -- 172.21.15.163:0/4035927749 learned_addr learned my addr 172.21.15.163:0/4035927749 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:46.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc4a9f640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc0140070 0x7fbcc006de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:46.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc5aa1640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcc0142d50 0x7fbcc015bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:46.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc52a0640 1 -- 172.21.15.163:0/4035927749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcc0142d50 msgr2=0x7fbcc015bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:46.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc52a0640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcc0142d50 0x7fbcc015bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:46.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc52a0640 1 -- 172.21.15.163:0/4035927749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc0140070 msgr2=0x7fbcc006de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:46.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc52a0640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc0140070 0x7fbcc006de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:46.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc52a0640 1 -- 172.21.15.163:0/4035927749 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc01437d0 con 0x7fbcc00722f0 2024-05-08T06:37:46.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.016+0000 7fbcc5aa1640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcc0142d50 0x7fbcc015bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:46.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.017+0000 7fbcc52a0640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcc00722f0 0x7fbcc01f8610 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc02a0f0 tx=0x7fbcbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:46.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.017+0000 7fbcb5ffb640 1 -- 172.21.15.163:0/4035927749 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc041070 con 0x7fbcc00722f0 2024-05-08T06:37:46.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.017+0000 7fbcb5ffb640 1 -- 172.21.15.163:0/4035927749 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcbc03c670 con 0x7fbcc00722f0 2024-05-08T06:37:46.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.017+0000 7fbcb5ffb640 1 -- 172.21.15.163:0/4035927749 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc037040 con 0x7fbcc00722f0 2024-05-08T06:37:46.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.017+0000 7fbcc7504640 1 -- 172.21.15.163:0/4035927749 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc01f6b80 con 0x7fbcc00722f0 2024-05-08T06:37:46.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.017+0000 7fbcc7504640 1 -- 172.21.15.163:0/4035927749 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc0143d50 con 0x7fbcc00722f0 2024-05-08T06:37:46.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.018+0000 7fbcc7504640 1 -- 172.21.15.163:0/4035927749 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc006a550 con 0x7fbcc00722f0 2024-05-08T06:37:46.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.018+0000 7fbcb5ffb640 1 -- 172.21.15.163:0/4035927749 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbcbc03a070 con 0x7fbcc00722f0 2024-05-08T06:37:46.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.019+0000 7fbcb5ffb640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fbca40762d0 0x7fbca4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:46.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.019+0000 7fbcb5ffb640 1 -- 172.21.15.163:0/4035927749 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fbcbc0b7640 con 0x7fbcc00722f0 2024-05-08T06:37:46.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.019+0000 7fbcc4a9f640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fbca40762d0 0x7fbca4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:46.024 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.019+0000 7fbcc4a9f640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fbca40762d0 0x7fbca4078790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbcc013f670 tx=0x7fbcb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:46.024 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.023+0000 7fbcb5ffb640 1 -- 172.21.15.163:0/4035927749 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbcbc081240 con 0x7fbcc00722f0 2024-05-08T06:37:46.206 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.207+0000 7fbcc7504640 1 -- 172.21.15.163:0/4035927749 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbcc00b5a40 con 0x7fbcc00722f0 2024-05-08T06:37:46.207 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.208+0000 7fbcb5ffb640 1 -- 172.21.15.163:0/4035927749 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13176 (secure 0 0 0) 0x7fbcbc080b40 con 0x7fbcc00722f0 2024-05-08T06:37:46.207 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:46.208 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":35,"fsid":"e639506e-cf71-49c6-ae4d-5af4601c36e0","created":"2024-05-08T06:36:17.750539+0000","modified":"2024-05-08T06:37:42.780778+0000","last_up_change":"2024-05-08T06:36:29.098545+0000","last_in_change":"2024-05-08T06:36:20.012645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:36:30.111982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7592d5ac-4220-44b4-9465-3cc63b2b47fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6801","nonce":3059241761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6803","nonce":3059241761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6807","nonce":3059241761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3059241761},{"type":"v1","addr":"172.21.15.163:6805","nonce":3059241761}]},"public_addr":"172.21.15.163:6801/3059241761","cluster_addr":"172.21.15.163:6803/3059241761","heartbeat_back_addr":"172.21.15.163:6807/3059241761","heartbeat_front_addr":"172.21.15.163:6805/3059241761","state":["exists","up"]},{"osd":1,"uuid":"181f5fb9-4a94-4fa1-b8c0-79500f415678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6825","nonce":2322245857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6827","nonce":2322245857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6831","nonce":2322245857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2322245857},{"type":"v1","addr":"172.21.15.163:6829","nonce":2322245857}]},"public_addr":"172.21.15.163:6825/2322245857","cluster_addr":"172.21.15.163:6827/2322245857","heartbeat_back_addr":"172.21.15.163:6831/2322245857","heartbeat_front_addr":"172.21.15.163:6829/2322245857","state":["exists","up"]},{"osd":2,"uuid":"5a16ce86-0f45-447c-9b9f-50cfc689a750","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6809","nonce":601639604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6811","nonce":601639604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6815","nonce":601639604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":601639604},{"type":"v1","addr":"172.21.15.163:6813","nonce":601639604}]},"public_addr":"172.21.15.163:6809/601639604","cluster_addr":"172.21.15.163:6811/601639604","heartbeat_back_addr":"172.21.15.163:6815/601639604","heartbeat_front_addr":"172.21.15.163:6813/601639604","state":["exists","up"]},{"osd":3,"uuid":"c0b95302-e771-4514-9347-56dad6b5b6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6817","nonce":127592521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6819","nonce":127592521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6823","nonce":127592521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":127592521},{"type":"v1","addr":"172.21.15.163:6821","nonce":127592521}]},"public_addr":"172.21.15.163:6817/127592521","cluster_addr":"172.21.15.163:6819/127592521","heartbeat_back_addr":"172.21.15.163:6823/127592521","heartbeat_front_addr":"172.21.15.163:6821/127592521","state":["exists","up"]},{"osd":4,"uuid":"78c7a01f-b6b3-411a-8f8d-a82da908ad37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6825","nonce":272496966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6827","nonce":272496966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6831","nonce":272496966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":272496966},{"type":"v1","addr":"172.21.15.192:6829","nonce":272496966}]},"public_addr":"172.21.15.192:6825/272496966","cluster_addr":"172.21.15.192:6827/272496966","heartbeat_back_addr":"172.21.15.192:6831/272496966","heartbeat_front_addr":"172.21.15.192:6829/272496966","state":["exists","up"]},{"osd":5,"uuid":"5729f316-c353-468c-929b-44184879d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6801","nonce":2911825781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6803","nonce":2911825781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6807","nonce":2911825781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2911825781},{"type":"v1","addr":"172.21.15.192:6805","nonce":2911825781}]},"public_addr":"172.21.15.192:6801/2911825781","cluster_addr":"172.21.15.192:6803/2911825781","heartbeat_back_addr":"172.21.15.192:6807/2911825781","heartbeat_front_addr":"172.21.15.192:6805/2911825781","state":["exists","up"]},{"osd":6,"uuid":"63e9d47f-50fe-4ac6-b7bf-de0256fdccfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6817","nonce":1074108675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6819","nonce":1074108675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6823","nonce":1074108675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1074108675},{"type":"v1","addr":"172.21.15.192:6821","nonce":1074108675}]},"public_addr":"172.21.15.192:6817/1074108675","cluster_addr":"172.21.15.192:6819/1074108675","heartbeat_back_addr":"172.21.15.192:6823/1074108675","heartbeat_front_addr":"172.21.15.192:6821/1074108675","state":["exists","up"]},{"osd":7,"uuid":"b1615978-384e-436f-a343-60cc5bd9d058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6809","nonce":1747004047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6811","nonce":1747004047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6815","nonce":1747004047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1747004047},{"type":"v1","addr":"172.21.15.192:6813","nonce":1747004047}]},"public_addr":"172.21.15.192:6809/1747004047","cluster_addr":"172.21.15.192:6811/1747004047","heartbeat_back_addr":"172.21.15.192:6815/1747004047","heartbeat_front_addr":"172.21.15.192:6813/1747004047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.053457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.173158+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.143997+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.132657+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.199477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.118577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.164505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:36:25.172992+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/3423234723":"2024-05-09T06:37:39.771867+0000","172.21.15.163:6836/3423234723":"2024-05-09T06:37:39.771867+0000","172.21.15.163:6837/880835577":"2024-05-09T06:37:31.216100+0000","172.21.15.163:6836/880835577":"2024-05-09T06:37:31.216100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:37:46.210 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.210+0000 7fbcc7504640 1 -- 172.21.15.163:0/4035927749 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fbca40762d0 msgr2=0x7fbca4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:46.210 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.210+0000 7fbcc7504640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fbca40762d0 0x7fbca4078790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbcc013f670 tx=0x7fbcb0008040 comp rx=0 tx=0).stop 2024-05-08T06:37:46.210 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.210+0000 7fbcc7504640 1 -- 172.21.15.163:0/4035927749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcc00722f0 msgr2=0x7fbcc01f8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:46.210 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.210+0000 7fbcc7504640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcc00722f0 0x7fbcc01f8610 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc02a0f0 tx=0x7fbcbc0046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:46.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.210+0000 7fbcc7504640 1 -- 172.21.15.163:0/4035927749 shutdown_connections 2024-05-08T06:37:46.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.210+0000 7fbcc7504640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fbca40762d0 0x7fbca4078790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:46.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.210+0000 7fbcc7504640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcc0142d50 0x7fbcc015bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:46.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.210+0000 7fbcc7504640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc0140070 0x7fbcc006de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:46.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.210+0000 7fbcc7504640 1 --2- 172.21.15.163:0/4035927749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcc00722f0 0x7fbcc01f8610 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:46.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.210+0000 7fbcc7504640 1 -- 172.21.15.163:0/4035927749 >> 172.21.15.163:0/4035927749 conn(0x7fbcc00b4da0 msgr2=0x7fbcc01608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:46.211 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.210+0000 7fbcc7504640 1 -- 172.21.15.163:0/4035927749 shutdown_connections 2024-05-08T06:37:46.212 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:46.210+0000 7fbcc7504640 1 -- 172.21.15.163:0/4035927749 wait complete. 2024-05-08T06:37:47.221 INFO:tasks.ceph:Scrubbing osd.0 2024-05-08T06:37:47.222 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-08T06:37:47.328 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.329+0000 7fa384ec4640 1 Processor -- start 2024-05-08T06:37:47.328 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.329+0000 7fa384ec4640 1 -- start start 2024-05-08T06:37:47.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.330+0000 7fa384ec4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa38015c7d0 0x7fa38006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.330+0000 7fa384ec4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3800b4f80 con 0x7fa3800b49f0 2024-05-08T06:37:47.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.330+0000 7fa384ec4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3800b5260 con 0x7fa38006f010 2024-05-08T06:37:47.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.330+0000 7fa384ec4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa38015d070 con 0x7fa38015c7d0 2024-05-08T06:37:47.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.330+0000 7fa37e59c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa38015c7d0 0x7fa38006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.330+0000 7fa37ed9d640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa3800b49f0 0x7fa38015db50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:41662/0 (socket says 172.21.15.163:41662) 2024-05-08T06:37:47.330 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.330+0000 7fa37ed9d640 1 -- 172.21.15.163:0/3066361305 learned_addr learned my addr 172.21.15.163:0/3066361305 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:47.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.331+0000 7fa37e59c640 1 -- 172.21.15.163:0/3066361305 >> v1:172.21.15.192:6789/0 conn(0x7fa38006f010 legacy=0x7fa3800b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.331+0000 7fa37e59c640 1 -- 172.21.15.163:0/3066361305 >> v1:172.21.15.163:6789/0 conn(0x7fa3800b49f0 legacy=0x7fa38015db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.331+0000 7fa37e59c640 1 -- 172.21.15.163:0/3066361305 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38015d350 con 0x7fa38015c7d0 2024-05-08T06:37:47.331 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.331+0000 7fa37dd9b640 1 -- 172.21.15.163:0/3066361305 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3880412053 0 0) 0x7fa3800b4f80 con 0x7fa3800b49f0 2024-05-08T06:37:47.332 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.331+0000 7fa37e59c640 1 --2- 172.21.15.163:0/3066361305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa38015c7d0 0x7fa38006ead0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa37000a940 tx=0x7fa37002b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15584d1cdafc78ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:47.333 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.331+0000 7fa37dd9b640 1 -- 172.21.15.163:0/3066361305 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37000b040 con 0x7fa38015c7d0 2024-05-08T06:37:47.333 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.331+0000 7fa37dd9b640 1 -- 172.21.15.163:0/3066361305 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa370030510 con 0x7fa38015c7d0 2024-05-08T06:37:47.333 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.331+0000 7fa37dd9b640 1 -- 172.21.15.163:0/3066361305 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3700308e0 con 0x7fa38015c7d0 2024-05-08T06:37:47.333 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.332+0000 7fa384ec4640 1 -- 172.21.15.163:0/3066361305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa38015c7d0 msgr2=0x7fa38006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.333 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.332+0000 7fa384ec4640 1 --2- 172.21.15.163:0/3066361305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa38015c7d0 0x7fa38006ead0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa37000a940 tx=0x7fa37002b5e0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.333 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.332+0000 7fa384ec4640 1 -- 172.21.15.163:0/3066361305 shutdown_connections 2024-05-08T06:37:47.333 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.332+0000 7fa384ec4640 1 --2- 172.21.15.163:0/3066361305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa38015c7d0 0x7fa38006ead0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.333 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.332+0000 7fa384ec4640 1 -- 172.21.15.163:0/3066361305 >> 172.21.15.163:0/3066361305 conn(0x7fa3800b2b40 msgr2=0x7fa3800b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:47.334 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.332+0000 7fa384ec4640 1 -- 172.21.15.163:0/3066361305 shutdown_connections 2024-05-08T06:37:47.334 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.332+0000 7fa384ec4640 1 -- 172.21.15.163:0/3066361305 wait complete. 2024-05-08T06:37:47.334 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.333+0000 7fa384ec4640 1 Processor -- start 2024-05-08T06:37:47.334 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.333+0000 7fa384ec4640 1 -- start start 2024-05-08T06:37:47.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.334+0000 7fa384ec4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa38006f010 0x7fa38006e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.334+0000 7fa384ec4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3800b49f0 0x7fa3801f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.334+0000 7fa384ec4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa38015c7d0 0x7fa3801f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.334+0000 7fa384ec4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa38015cbb0 con 0x7fa3800b49f0 2024-05-08T06:37:47.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.334+0000 7fa384ec4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa38006f420 con 0x7fa38006f010 2024-05-08T06:37:47.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.334+0000 7fa384ec4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa38015cd30 con 0x7fa38015c7d0 2024-05-08T06:37:47.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.334+0000 7fa377fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3800b49f0 0x7fa3801f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.334+0000 7fa377fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3800b49f0 0x7fa3801f6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33928/0 (socket says 172.21.15.163:33928) 2024-05-08T06:37:47.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.334+0000 7fa377fff640 1 -- 172.21.15.163:0/638394541 learned_addr learned my addr 172.21.15.163:0/638394541 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:47.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.334+0000 7fa37e59c640 1 --2- 172.21.15.163:0/638394541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa38006f010 0x7fa38006e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.335+0000 7fa377fff640 1 -- 172.21.15.163:0/638394541 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa38015c7d0 msgr2=0x7fa3801f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.335+0000 7fa377fff640 1 --2- 172.21.15.163:0/638394541 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa38015c7d0 0x7fa3801f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.335+0000 7fa377fff640 1 -- 172.21.15.163:0/638394541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa38006f010 msgr2=0x7fa38006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.338 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.335+0000 7fa377fff640 1 --2- 172.21.15.163:0/638394541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa38006f010 0x7fa38006e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.335+0000 7fa377fff640 1 -- 172.21.15.163:0/638394541 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3801558e0 con 0x7fa3800b49f0 2024-05-08T06:37:47.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.335+0000 7fa37e59c640 1 --2- 172.21.15.163:0/638394541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa38006f010 0x7fa38006e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:47.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.335+0000 7fa377fff640 1 --2- 172.21.15.163:0/638394541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3800b49f0 0x7fa3801f6740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa36800c400 tx=0x7fa36800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:47.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.335+0000 7fa376ffd640 1 -- 172.21.15.163:0/638394541 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368017070 con 0x7fa3800b49f0 2024-05-08T06:37:47.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.335+0000 7fa376ffd640 1 -- 172.21.15.163:0/638394541 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3680052d0 con 0x7fa3800b49f0 2024-05-08T06:37:47.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.335+0000 7fa376ffd640 1 -- 172.21.15.163:0/638394541 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368003830 con 0x7fa3800b49f0 2024-05-08T06:37:47.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.335+0000 7fa384ec4640 1 -- 172.21.15.163:0/638394541 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa380155bc0 con 0x7fa3800b49f0 2024-05-08T06:37:47.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.335+0000 7fa384ec4640 1 -- 172.21.15.163:0/638394541 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa380149a80 con 0x7fa3800b49f0 2024-05-08T06:37:47.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.336+0000 7fa384ec4640 1 -- 172.21.15.163:0/638394541 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa380149e80 con 0x7fa3800b49f0 2024-05-08T06:37:47.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.337+0000 7fa376ffd640 1 -- 172.21.15.163:0/638394541 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa36801a070 con 0x7fa3800b49f0 2024-05-08T06:37:47.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.337+0000 7fa376ffd640 1 --2- 172.21.15.163:0/638394541 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa3600762d0 0x7fa360078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.337+0000 7fa376ffd640 1 -- 172.21.15.163:0/638394541 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fa368096e00 con 0x7fa3800b49f0 2024-05-08T06:37:47.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.337+0000 7fa37e59c640 1 --2- 172.21.15.163:0/638394541 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa3600762d0 0x7fa360078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.337+0000 7fa376ffd640 1 --2- 172.21.15.163:0/638394541 >> [v2:172.21.15.163:6800/3059241761,v1:172.21.15.163:6801/3059241761] conn(0x7fa36007c230 0x7fa36007e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.337+0000 7fa376ffd640 1 -- 172.21.15.163:0/638394541 --> [v2:172.21.15.163:6800/3059241761,v1:172.21.15.163:6801/3059241761] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa36801dec0 con 0x7fa36007c230 2024-05-08T06:37:47.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.337+0000 7fa376ffd640 1 -- 172.21.15.163:0/638394541 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa368097290 con 0x7fa3800b49f0 2024-05-08T06:37:47.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.338+0000 7fa37ed9d640 1 --2- 172.21.15.163:0/638394541 >> [v2:172.21.15.163:6800/3059241761,v1:172.21.15.163:6801/3059241761] conn(0x7fa36007c230 0x7fa36007e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.338+0000 7fa37e59c640 1 --2- 172.21.15.163:0/638394541 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa3600762d0 0x7fa360078790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa370035000 tx=0x7fa37000c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:47.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.338+0000 7fa37ed9d640 1 --2- 172.21.15.163:0/638394541 >> [v2:172.21.15.163:6800/3059241761,v1:172.21.15.163:6801/3059241761] conn(0x7fa36007c230 0x7fa36007e650 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:47.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.338+0000 7fa376ffd640 1 -- 172.21.15.163:0/638394541 <== osd.0 v2:172.21.15.163:6800/3059241761 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa36801dec0 con 0x7fa36007c230 2024-05-08T06:37:47.350 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.351+0000 7fa384ec4640 1 -- 172.21.15.163:0/638394541 --> [v2:172.21.15.163:6800/3059241761,v1:172.21.15.163:6801/3059241761] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa38006a550 con 0x7fa36007c230 2024-05-08T06:37:47.351 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.352+0000 7fa376ffd640 1 -- 172.21.15.163:0/638394541 <== osd.0 v2:172.21.15.163:6800/3059241761 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa38006a550 con 0x7fa36007c230 2024-05-08T06:37:47.351 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:37:47.351 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T06:37:47.351 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:37:47.352 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.352+0000 7fa384ec4640 1 -- 172.21.15.163:0/638394541 >> [v2:172.21.15.163:6800/3059241761,v1:172.21.15.163:6801/3059241761] conn(0x7fa36007c230 msgr2=0x7fa36007e650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.352 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.352+0000 7fa384ec4640 1 --2- 172.21.15.163:0/638394541 >> [v2:172.21.15.163:6800/3059241761,v1:172.21.15.163:6801/3059241761] conn(0x7fa36007c230 0x7fa36007e650 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.353 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.352+0000 7fa384ec4640 1 -- 172.21.15.163:0/638394541 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa3600762d0 msgr2=0x7fa360078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.353 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.352+0000 7fa384ec4640 1 --2- 172.21.15.163:0/638394541 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa3600762d0 0x7fa360078790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa370035000 tx=0x7fa37000c040 comp rx=0 tx=0).stop 2024-05-08T06:37:47.353 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.352+0000 7fa384ec4640 1 -- 172.21.15.163:0/638394541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3800b49f0 msgr2=0x7fa3801f6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.353 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.352+0000 7fa384ec4640 1 --2- 172.21.15.163:0/638394541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3800b49f0 0x7fa3801f6740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa36800c400 tx=0x7fa36800c8c0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.353 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.352+0000 7fa37ed9d640 1 -- 172.21.15.163:0/638394541 reap_dead start 2024-05-08T06:37:47.353 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.352+0000 7fa384ec4640 1 -- 172.21.15.163:0/638394541 shutdown_connections 2024-05-08T06:37:47.353 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.352+0000 7fa384ec4640 1 -- 172.21.15.163:0/638394541 >> 172.21.15.163:0/638394541 conn(0x7fa3800b2b40 msgr2=0x7fa38014d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:47.353 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.352+0000 7fa384ec4640 1 -- 172.21.15.163:0/638394541 shutdown_connections 2024-05-08T06:37:47.354 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.352+0000 7fa384ec4640 1 -- 172.21.15.163:0/638394541 wait complete. 2024-05-08T06:37:47.362 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-08T06:37:47.465 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.466+0000 7fbe4341d640 1 Processor -- start 2024-05-08T06:37:47.465 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.466+0000 7fbe4341d640 1 -- start start 2024-05-08T06:37:47.466 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.467+0000 7fbe4341d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbe3c0af290 0x7fbe3c0b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.467+0000 7fbe4341d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe3c0b0770 0x7fbe3c0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.467+0000 7fbe4341d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe3c0b74c0 con 0x7fbe3c15f450 2024-05-08T06:37:47.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.467+0000 7fbe4341d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fbe3c0af800 con 0x7fbe3c0af290 2024-05-08T06:37:47.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.467+0000 7fbe4341d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbe3c0b62d0 con 0x7fbe3c0b0770 2024-05-08T06:37:47.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.467+0000 7fbe409b8640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbe3c0af290 0x7fbe3c0b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.467+0000 7fbe409b8640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbe3c0af290 0x7fbe3c0b7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33942/0 (socket says 172.21.15.163:33942) 2024-05-08T06:37:47.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.467+0000 7fbe409b8640 1 -- 172.21.15.163:0/237371936 learned_addr learned my addr 172.21.15.163:0/237371936 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:47.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.467+0000 7fbe409b8640 1 -- 172.21.15.163:0/237371936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe3c0b0770 msgr2=0x7fbe3c0b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:47.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.467+0000 7fbe409b8640 1 --2- 172.21.15.163:0/237371936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe3c0b0770 0x7fbe3c0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.467+0000 7fbe409b8640 1 -- 172.21.15.163:0/237371936 >> v1:172.21.15.192:6789/0 conn(0x7fbe3c15f450 legacy=0x7fbe3c0b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:47.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.467+0000 7fbe409b8640 1 -- 172.21.15.163:0/237371936 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3c0b65b0 con 0x7fbe3c0af290 2024-05-08T06:37:47.468 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.468+0000 7fbe409b8640 1 --2- 172.21.15.163:0/237371936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe3c0af290 0x7fbe3c0b7780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009870 tx=0x7fbe2c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176817410800aaf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:47.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.468+0000 7fbe33fff640 1 -- 172.21.15.163:0/237371936 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c037070 con 0x7fbe3c0af290 2024-05-08T06:37:47.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.468+0000 7fbe33fff640 1 -- 172.21.15.163:0/237371936 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe2c032440 con 0x7fbe3c0af290 2024-05-08T06:37:47.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.468+0000 7fbe33fff640 1 -- 172.21.15.163:0/237371936 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c032810 con 0x7fbe3c0af290 2024-05-08T06:37:47.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.469+0000 7fbe4341d640 1 -- 172.21.15.163:0/237371936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe3c0af290 msgr2=0x7fbe3c0b7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.469+0000 7fbe4341d640 1 --2- 172.21.15.163:0/237371936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe3c0af290 0x7fbe3c0b7780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009870 tx=0x7fbe2c02a1c0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.469+0000 7fbe4341d640 1 -- 172.21.15.163:0/237371936 shutdown_connections 2024-05-08T06:37:47.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.469+0000 7fbe4341d640 1 --2- 172.21.15.163:0/237371936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe3c0b0770 0x7fbe3c0b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.469+0000 7fbe4341d640 1 --2- 172.21.15.163:0/237371936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe3c0af290 0x7fbe3c0b7780 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.469+0000 7fbe4341d640 1 -- 172.21.15.163:0/237371936 >> 172.21.15.163:0/237371936 conn(0x7fbe3c073b40 msgr2=0x7fbe3c06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:47.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.469+0000 7fbe4341d640 1 -- 172.21.15.163:0/237371936 shutdown_connections 2024-05-08T06:37:47.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.469+0000 7fbe4341d640 1 -- 172.21.15.163:0/237371936 wait complete. 2024-05-08T06:37:47.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.470+0000 7fbe4341d640 1 Processor -- start 2024-05-08T06:37:47.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.470+0000 7fbe4341d640 1 -- start start 2024-05-08T06:37:47.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe4341d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe3c0af290 0x7fbe3c13e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe4341d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe3c0b0770 0x7fbe3c158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe4341d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe3c15f450 0x7fbe3c0b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe4341d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c0b6070 con 0x7fbe3c0b0770 2024-05-08T06:37:47.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe4341d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c0b5ef0 con 0x7fbe3c15f450 2024-05-08T06:37:47.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe4341d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbe3c0b03d0 con 0x7fbe3c0af290 2024-05-08T06:37:47.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe409b8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe3c0b0770 0x7fbe3c158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe409b8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe3c0b0770 0x7fbe3c158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33948/0 (socket says 172.21.15.163:33948) 2024-05-08T06:37:47.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe409b8640 1 -- 172.21.15.163:0/1342790574 learned_addr learned my addr 172.21.15.163:0/1342790574 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:47.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe409b8640 1 -- 172.21.15.163:0/1342790574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe3c0af290 msgr2=0x7fbe3c13e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe409b8640 1 --2- 172.21.15.163:0/1342790574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe3c0af290 0x7fbe3c13e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe409b8640 1 -- 172.21.15.163:0/1342790574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe3c15f450 msgr2=0x7fbe3c0b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe409b8640 1 --2- 172.21.15.163:0/1342790574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe3c15f450 0x7fbe3c0b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe409b8640 1 -- 172.21.15.163:0/1342790574 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3c13e950 con 0x7fbe3c0b0770 2024-05-08T06:37:47.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe409b8640 1 --2- 172.21.15.163:0/1342790574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe3c0b0770 0x7fbe3c158690 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009870 tx=0x7fbe2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:47.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe31ffb640 1 -- 172.21.15.163:0/1342790574 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c041070 con 0x7fbe3c0b0770 2024-05-08T06:37:47.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe31ffb640 1 -- 172.21.15.163:0/1342790574 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe2c03c870 con 0x7fbe3c0b0770 2024-05-08T06:37:47.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe4341d640 1 -- 172.21.15.163:0/1342790574 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3c1f85d0 con 0x7fbe3c0b0770 2024-05-08T06:37:47.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.471+0000 7fbe31ffb640 1 -- 172.21.15.163:0/1342790574 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c037040 con 0x7fbe3c0b0770 2024-05-08T06:37:47.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.472+0000 7fbe4341d640 1 -- 172.21.15.163:0/1342790574 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3c1f8b90 con 0x7fbe3c0b0770 2024-05-08T06:37:47.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.473+0000 7fbe4341d640 1 -- 172.21.15.163:0/1342790574 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe3c06a4b0 con 0x7fbe3c0b0770 2024-05-08T06:37:47.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.473+0000 7fbe31ffb640 1 -- 172.21.15.163:0/1342790574 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbe2c044070 con 0x7fbe3c0b0770 2024-05-08T06:37:47.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.474+0000 7fbe31ffb640 1 --2- 172.21.15.163:0/1342790574 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fbe20076300 0x7fbe200787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.474+0000 7fbe411b9640 1 --2- 172.21.15.163:0/1342790574 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fbe20076300 0x7fbe200787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.474+0000 7fbe31ffb640 1 -- 172.21.15.163:0/1342790574 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fbe2c0b7e10 con 0x7fbe3c0b0770 2024-05-08T06:37:47.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.474+0000 7fbe411b9640 1 --2- 172.21.15.163:0/1342790574 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fbe20076300 0x7fbe200787c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbe38006fd0 tx=0x7fbe38006bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:47.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.478+0000 7fbe31ffb640 1 -- 172.21.15.163:0/1342790574 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbe2c081a10 con 0x7fbe3c0b0770 2024-05-08T06:37:47.658 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.659+0000 7fbe4341d640 1 -- 172.21.15.163:0/1342790574 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fbe3c0b7130 con 0x7fbe20076300 2024-05-08T06:37:47.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.659+0000 7fbe31ffb640 1 -- 172.21.15.163:0/1342790574 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbe3c0b7130 con 0x7fbe20076300 2024-05-08T06:37:47.659 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 0 to deep-scrub 2024-05-08T06:37:47.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.661+0000 7fbe4341d640 1 -- 172.21.15.163:0/1342790574 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fbe20076300 msgr2=0x7fbe200787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.661+0000 7fbe4341d640 1 --2- 172.21.15.163:0/1342790574 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fbe20076300 0x7fbe200787c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbe38006fd0 tx=0x7fbe38006bc0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.661+0000 7fbe4341d640 1 -- 172.21.15.163:0/1342790574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe3c0b0770 msgr2=0x7fbe3c158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.661+0000 7fbe4341d640 1 --2- 172.21.15.163:0/1342790574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe3c0b0770 0x7fbe3c158690 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009870 tx=0x7fbe2c0046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.661+0000 7fbe4341d640 1 -- 172.21.15.163:0/1342790574 shutdown_connections 2024-05-08T06:37:47.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.661+0000 7fbe4341d640 1 --2- 172.21.15.163:0/1342790574 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fbe20076300 0x7fbe200787c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.661+0000 7fbe4341d640 1 --2- 172.21.15.163:0/1342790574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe3c15f450 0x7fbe3c0b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.661+0000 7fbe4341d640 1 --2- 172.21.15.163:0/1342790574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe3c0b0770 0x7fbe3c158690 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.661+0000 7fbe4341d640 1 --2- 172.21.15.163:0/1342790574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe3c0af290 0x7fbe3c13e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.661+0000 7fbe4341d640 1 -- 172.21.15.163:0/1342790574 >> 172.21.15.163:0/1342790574 conn(0x7fbe3c073b40 msgr2=0x7fbe3c06d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:47.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.661+0000 7fbe4341d640 1 -- 172.21.15.163:0/1342790574 shutdown_connections 2024-05-08T06:37:47.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.661+0000 7fbe4341d640 1 -- 172.21.15.163:0/1342790574 wait complete. 2024-05-08T06:37:47.671 INFO:tasks.ceph:Scrubbing osd.1 2024-05-08T06:37:47.672 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-08T06:37:47.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.777+0000 7f571cd89640 1 Processor -- start 2024-05-08T06:37:47.776 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.777+0000 7f571cd89640 1 -- start start 2024-05-08T06:37:47.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.778+0000 7f571cd89640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5718070e50 0x7f5718071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.778+0000 7f571cd89640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57180d3b60 0x7f571806de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.778+0000 7f571cd89640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57180b7b20 con 0x7f57180cf5a0 2024-05-08T06:37:47.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.778+0000 7f571cd89640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f57180b82b0 con 0x7f5718070e50 2024-05-08T06:37:47.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.778+0000 7f571cd89640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5718070900 con 0x7f57180d3b60 2024-05-08T06:37:47.778 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.778+0000 7f571659c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5718070e50 0x7f5718071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.778+0000 7f571659c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5718070e50 0x7f5718071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33958/0 (socket says 172.21.15.163:33958) 2024-05-08T06:37:47.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.778+0000 7f571659c640 1 -- 172.21.15.163:0/145612403 learned_addr learned my addr 172.21.15.163:0/145612403 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:47.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.778+0000 7f571659c640 1 -- 172.21.15.163:0/145612403 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57180d3b60 msgr2=0x7f571806de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:47.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.778+0000 7f571659c640 1 --2- 172.21.15.163:0/145612403 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57180d3b60 0x7f571806de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.778+0000 7f571659c640 1 -- 172.21.15.163:0/145612403 >> v1:172.21.15.192:6789/0 conn(0x7f57180cf5a0 legacy=0x7f571806e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:47.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.778+0000 7f571659c640 1 -- 172.21.15.163:0/145612403 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57180aff30 con 0x7f5718070e50 2024-05-08T06:37:47.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.779+0000 7f571659c640 1 --2- 172.21.15.163:0/145612403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5718070e50 0x7f5718071230 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5708009870 tx=0x7f570802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e78b65394607b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:47.779 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.779+0000 7f571559a640 1 -- 172.21.15.163:0/145612403 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708037070 con 0x7f5718070e50 2024-05-08T06:37:47.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.779+0000 7f571559a640 1 -- 172.21.15.163:0/145612403 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5708032440 con 0x7f5718070e50 2024-05-08T06:37:47.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.779+0000 7f571559a640 1 -- 172.21.15.163:0/145612403 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708032810 con 0x7f5718070e50 2024-05-08T06:37:47.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.779+0000 7f571cd89640 1 -- 172.21.15.163:0/145612403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5718070e50 msgr2=0x7f5718071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.779+0000 7f571cd89640 1 --2- 172.21.15.163:0/145612403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5718070e50 0x7f5718071230 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5708009870 tx=0x7f570802a1c0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.780+0000 7f571cd89640 1 -- 172.21.15.163:0/145612403 shutdown_connections 2024-05-08T06:37:47.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.780+0000 7f571cd89640 1 --2- 172.21.15.163:0/145612403 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57180d3b60 0x7f571806de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.780+0000 7f571cd89640 1 --2- 172.21.15.163:0/145612403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5718070e50 0x7f5718071230 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.780+0000 7f571cd89640 1 -- 172.21.15.163:0/145612403 >> 172.21.15.163:0/145612403 conn(0x7f57180d3040 msgr2=0x7f57180d3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:47.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.780+0000 7f571cd89640 1 -- 172.21.15.163:0/145612403 shutdown_connections 2024-05-08T06:37:47.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.780+0000 7f571cd89640 1 -- 172.21.15.163:0/145612403 wait complete. 2024-05-08T06:37:47.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.781+0000 7f571cd89640 1 Processor -- start 2024-05-08T06:37:47.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.781+0000 7f571cd89640 1 -- start start 2024-05-08T06:37:47.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f571cd89640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5718070e50 0x7f57181ea110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f571cd89640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57180cf5a0 0x7f57180b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f571cd89640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57180d3b60 0x7f57180b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f571cd89640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5718071a20 con 0x7f57180d3b60 2024-05-08T06:37:47.785 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f571cd89640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f57180b7e20 con 0x7f57180cf5a0 2024-05-08T06:37:47.785 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f571cd89640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5718071d00 con 0x7f5718070e50 2024-05-08T06:37:47.785 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f5716d9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57180d3b60 0x7f57180b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.785 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f5716d9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57180d3b60 0x7f57180b11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33972/0 (socket says 172.21.15.163:33972) 2024-05-08T06:37:47.785 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f5716d9d640 1 -- 172.21.15.163:0/2519422200 learned_addr learned my addr 172.21.15.163:0/2519422200 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:47.785 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f571659c640 1 --2- 172.21.15.163:0/2519422200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5718070e50 0x7f57181ea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.785 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f5715d9b640 1 --2- 172.21.15.163:0/2519422200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57180cf5a0 0x7f57180b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.785 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f5716d9d640 1 -- 172.21.15.163:0/2519422200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5718070e50 msgr2=0x7f57181ea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.786 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f5716d9d640 1 --2- 172.21.15.163:0/2519422200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5718070e50 0x7f57181ea110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.786 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f5716d9d640 1 -- 172.21.15.163:0/2519422200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57180cf5a0 msgr2=0x7f57180b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.786 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f5716d9d640 1 --2- 172.21.15.163:0/2519422200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57180cf5a0 0x7f57180b0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.786 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f5716d9d640 1 -- 172.21.15.163:0/2519422200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57180cea10 con 0x7f57180d3b60 2024-05-08T06:37:47.786 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f5716d9d640 1 --2- 172.21.15.163:0/2519422200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57180d3b60 0x7f57180b11d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f571000c780 tx=0x7f571000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:47.786 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f5706ffd640 1 -- 172.21.15.163:0/2519422200 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710017070 con 0x7f57180d3b60 2024-05-08T06:37:47.786 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.782+0000 7f5706ffd640 1 -- 172.21.15.163:0/2519422200 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5710003da0 con 0x7f57180d3b60 2024-05-08T06:37:47.786 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.783+0000 7f5706ffd640 1 -- 172.21.15.163:0/2519422200 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57100052b0 con 0x7f57180d3b60 2024-05-08T06:37:47.786 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.783+0000 7f571cd89640 1 -- 172.21.15.163:0/2519422200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57180cecf0 con 0x7f57180d3b60 2024-05-08T06:37:47.787 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.783+0000 7f571cd89640 1 -- 172.21.15.163:0/2519422200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57180cdc60 con 0x7f57180d3b60 2024-05-08T06:37:47.787 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.783+0000 7f571cd89640 1 -- 172.21.15.163:0/2519422200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57180cf210 con 0x7f57180d3b60 2024-05-08T06:37:47.787 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.784+0000 7f5706ffd640 1 -- 172.21.15.163:0/2519422200 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f571001a070 con 0x7f57180d3b60 2024-05-08T06:37:47.788 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.785+0000 7f5706ffd640 1 --2- 172.21.15.163:0/2519422200 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f56f80762d0 0x7f56f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.788 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.785+0000 7f5706ffd640 1 -- 172.21.15.163:0/2519422200 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f57100970a0 con 0x7f57180d3b60 2024-05-08T06:37:47.788 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.785+0000 7f571659c640 1 --2- 172.21.15.163:0/2519422200 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f56f80762d0 0x7f56f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.788 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.785+0000 7f5706ffd640 1 --2- 172.21.15.163:0/2519422200 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x7f56f807c230 0x7f56f807e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.785+0000 7f5706ffd640 1 -- 172.21.15.163:0/2519422200 --> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5710020ec0 con 0x7f56f807c230 2024-05-08T06:37:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.785+0000 7f5706ffd640 1 -- 172.21.15.163:0/2519422200 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5710097530 con 0x7f57180d3b60 2024-05-08T06:37:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.785+0000 7f5715d9b640 1 --2- 172.21.15.163:0/2519422200 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x7f56f807c230 0x7f56f807e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.785+0000 7f571659c640 1 --2- 172.21.15.163:0/2519422200 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f56f80762d0 0x7f56f8078790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5708034000 tx=0x7f570802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.785+0000 7f5715d9b640 1 --2- 172.21.15.163:0/2519422200 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x7f56f807c230 0x7f56f807e650 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.786+0000 7f5706ffd640 1 -- 172.21.15.163:0/2519422200 <== osd.1 v2:172.21.15.163:6824/2322245857 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5710020ec0 con 0x7f56f807c230 2024-05-08T06:37:47.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.799+0000 7f571cd89640 1 -- 172.21.15.163:0/2519422200 --> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f57181f5310 con 0x7f56f807c230 2024-05-08T06:37:47.798 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.799+0000 7f5706ffd640 1 -- 172.21.15.163:0/2519422200 <== osd.1 v2:172.21.15.163:6824/2322245857 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f57181f5310 con 0x7f56f807c230 2024-05-08T06:37:47.799 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:37:47.799 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T06:37:47.799 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:37:47.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.799+0000 7f571cd89640 1 -- 172.21.15.163:0/2519422200 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x7f56f807c230 msgr2=0x7f56f807e650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.799+0000 7f571cd89640 1 --2- 172.21.15.163:0/2519422200 >> [v2:172.21.15.163:6824/2322245857,v1:172.21.15.163:6825/2322245857] conn(0x7f56f807c230 0x7f56f807e650 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.799+0000 7f571cd89640 1 -- 172.21.15.163:0/2519422200 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f56f80762d0 msgr2=0x7f56f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.799+0000 7f571cd89640 1 --2- 172.21.15.163:0/2519422200 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f56f80762d0 0x7f56f8078790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5708034000 tx=0x7f570802e000 comp rx=0 tx=0).stop 2024-05-08T06:37:47.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.799+0000 7f571cd89640 1 -- 172.21.15.163:0/2519422200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57180d3b60 msgr2=0x7f57180b11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.799+0000 7f571cd89640 1 --2- 172.21.15.163:0/2519422200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57180d3b60 0x7f57180b11d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f571000c780 tx=0x7f571000cc40 comp rx=0 tx=0).stop 2024-05-08T06:37:47.800 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.799+0000 7f5716d9d640 1 -- 172.21.15.163:0/2519422200 reap_dead start 2024-05-08T06:37:47.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.800+0000 7f571cd89640 1 -- 172.21.15.163:0/2519422200 shutdown_connections 2024-05-08T06:37:47.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.800+0000 7f571cd89640 1 -- 172.21.15.163:0/2519422200 >> 172.21.15.163:0/2519422200 conn(0x7f57180d3040 msgr2=0x7f571806e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:47.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.800+0000 7f571cd89640 1 -- 172.21.15.163:0/2519422200 shutdown_connections 2024-05-08T06:37:47.801 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.800+0000 7f571cd89640 1 -- 172.21.15.163:0/2519422200 wait complete. 2024-05-08T06:37:47.810 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-08T06:37:47.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.912+0000 7f04730a0640 1 Processor -- start 2024-05-08T06:37:47.912 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.912+0000 7f04730a0640 1 -- start start 2024-05-08T06:37:47.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.913+0000 7f04730a0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046c15f500 0x7f046c06f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.913+0000 7f04730a0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f046c0b7140 0x7f046c073b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.913+0000 7f04730a0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f046c0b3f60 0x7f046c0b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.913+0000 7f04730a0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f046c0b18f0 con 0x7f046c0b3f60 2024-05-08T06:37:47.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.913+0000 7f04730a0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f046c0b4910 con 0x7f046c0b7140 2024-05-08T06:37:47.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.913+0000 7f04730a0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f046c06fa10 con 0x7f046c15f500 2024-05-08T06:37:47.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.913+0000 7f047163d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f046c0b3f60 0x7f046c0b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.914+0000 7f047163d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f046c0b3f60 0x7f046c0b43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33988/0 (socket says 172.21.15.163:33988) 2024-05-08T06:37:47.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.914+0000 7f047163d640 1 -- 172.21.15.163:0/3677602787 learned_addr learned my addr 172.21.15.163:0/3677602787 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:47.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.914+0000 7f0470e3c640 1 --2- 172.21.15.163:0/3677602787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046c15f500 0x7f046c06f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.914+0000 7f046bfff640 1 --2- 172.21.15.163:0/3677602787 >> v2:172.21.15.192:3300/0 conn(0x7f046c0b7140 0x7f046c073b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.914+0000 7f047163d640 1 -- 172.21.15.163:0/3677602787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046c15f500 msgr2=0x7f046c06f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:47.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.914+0000 7f047163d640 1 --2- 172.21.15.163:0/3677602787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046c15f500 0x7f046c06f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.914+0000 7f047163d640 1 -- 172.21.15.163:0/3677602787 >> v2:172.21.15.192:3300/0 conn(0x7f046c0b7140 msgr2=0x7f046c073b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:47.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.914+0000 7f047163d640 1 --2- 172.21.15.163:0/3677602787 >> v2:172.21.15.192:3300/0 conn(0x7f046c0b7140 0x7f046c073b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.914+0000 7f047163d640 1 -- 172.21.15.163:0/3677602787 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046c0b0fc0 con 0x7f046c0b3f60 2024-05-08T06:37:47.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.914+0000 7f047163d640 1 --2- 172.21.15.163:0/3677602787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f046c0b3f60 0x7f046c0b43d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f046c06d360 tx=0x7f045402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a52449e34e75b28 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:47.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.915+0000 7f046b7fe640 1 -- 172.21.15.163:0/3677602787 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454037070 con 0x7f046c0b3f60 2024-05-08T06:37:47.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.915+0000 7f046b7fe640 1 -- 172.21.15.163:0/3677602787 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0454032440 con 0x7f046c0b3f60 2024-05-08T06:37:47.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.915+0000 7f046b7fe640 1 -- 172.21.15.163:0/3677602787 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454032810 con 0x7f046c0b3f60 2024-05-08T06:37:47.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.915+0000 7f04730a0640 1 -- 172.21.15.163:0/3677602787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f046c0b3f60 msgr2=0x7f046c0b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.915+0000 7f04730a0640 1 --2- 172.21.15.163:0/3677602787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f046c0b3f60 0x7f046c0b43d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f046c06d360 tx=0x7f045402a1e0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.915+0000 7f04730a0640 1 -- 172.21.15.163:0/3677602787 shutdown_connections 2024-05-08T06:37:47.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.915+0000 7f04730a0640 1 --2- 172.21.15.163:0/3677602787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f046c0b3f60 0x7f046c0b43d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.915+0000 7f04730a0640 1 --2- 172.21.15.163:0/3677602787 >> v2:172.21.15.192:3300/0 conn(0x7f046c0b7140 0x7f046c073b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.915+0000 7f04730a0640 1 --2- 172.21.15.163:0/3677602787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046c15f500 0x7f046c06f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.915+0000 7f04730a0640 1 -- 172.21.15.163:0/3677602787 >> 172.21.15.163:0/3677602787 conn(0x7f046c06d610 msgr2=0x7f046c06d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:47.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.915+0000 7f04730a0640 1 -- 172.21.15.163:0/3677602787 shutdown_connections 2024-05-08T06:37:47.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.915+0000 7f04730a0640 1 -- 172.21.15.163:0/3677602787 wait complete. 2024-05-08T06:37:47.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.916+0000 7f04730a0640 1 Processor -- start 2024-05-08T06:37:47.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.916+0000 7f04730a0640 1 -- start start 2024-05-08T06:37:47.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.917+0000 7f04730a0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f046c0b3f60 0x7f046c160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.917+0000 7f04730a0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f046c0b7140 0x7f046c160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.917+0000 7f04730a0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046c15f500 0x7f046c1f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.917+0000 7f04730a0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f046c074050 con 0x7f046c0b7140 2024-05-08T06:37:47.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.917+0000 7f04730a0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f046c0b1580 con 0x7f046c0b3f60 2024-05-08T06:37:47.921 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.917+0000 7f04730a0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f046c15fc70 con 0x7f046c15f500 2024-05-08T06:37:47.922 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.917+0000 7f046bfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f046c0b7140 0x7f046c160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.922 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.917+0000 7f046bfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f046c0b7140 0x7f046c160b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33994/0 (socket says 172.21.15.163:33994) 2024-05-08T06:37:47.922 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.917+0000 7f046bfff640 1 -- 172.21.15.163:0/4150400350 learned_addr learned my addr 172.21.15.163:0/4150400350 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:47.922 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.917+0000 7f0470e3c640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f046c0b3f60 0x7f046c160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.922 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.917+0000 7f047163d640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046c15f500 0x7f046c1f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.922 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.918+0000 7f046bfff640 1 -- 172.21.15.163:0/4150400350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046c15f500 msgr2=0x7f046c1f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.922 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.918+0000 7f046bfff640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046c15f500 0x7f046c1f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.922 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.918+0000 7f046bfff640 1 -- 172.21.15.163:0/4150400350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f046c0b3f60 msgr2=0x7f046c160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:47.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.918+0000 7f046bfff640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f046c0b3f60 0x7f046c160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:47.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.918+0000 7f046bfff640 1 -- 172.21.15.163:0/4150400350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046c157690 con 0x7f046c0b7140 2024-05-08T06:37:47.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.918+0000 7f047163d640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046c15f500 0x7f046c1f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:47.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.918+0000 7f046bfff640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f046c0b7140 0x7f046c160b50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f045c007f20 tx=0x7f045c00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:47.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.918+0000 7f04697fa640 1 -- 172.21.15.163:0/4150400350 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c014620 con 0x7f046c0b7140 2024-05-08T06:37:47.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.918+0000 7f04697fa640 1 -- 172.21.15.163:0/4150400350 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f045c019070 con 0x7f046c0b7140 2024-05-08T06:37:47.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.918+0000 7f04730a0640 1 -- 172.21.15.163:0/4150400350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046c158cd0 con 0x7f046c0b7140 2024-05-08T06:37:47.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.918+0000 7f04697fa640 1 -- 172.21.15.163:0/4150400350 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c014a00 con 0x7f046c0b7140 2024-05-08T06:37:47.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.918+0000 7f04730a0640 1 -- 172.21.15.163:0/4150400350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f046c159290 con 0x7f046c0b7140 2024-05-08T06:37:47.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.919+0000 7f04730a0640 1 -- 172.21.15.163:0/4150400350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f046c06a4b0 con 0x7f046c0b7140 2024-05-08T06:37:47.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.920+0000 7f04697fa640 1 -- 172.21.15.163:0/4150400350 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f045c014ba0 con 0x7f046c0b7140 2024-05-08T06:37:47.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.920+0000 7f04697fa640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f044c0762d0 0x7f044c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:47.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.920+0000 7f04697fa640 1 -- 172.21.15.163:0/4150400350 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f045c097c00 con 0x7f046c0b7140 2024-05-08T06:37:47.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.920+0000 7f0470e3c640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f044c0762d0 0x7f044c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:47.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.920+0000 7f0470e3c640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f044c0762d0 0x7f044c078790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0464007920 tx=0x7f0464008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:47.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:47.924+0000 7f04697fa640 1 -- 172.21.15.163:0/4150400350 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f045c09e080 con 0x7f046c0b7140 2024-05-08T06:37:48.106 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.107+0000 7f04730a0640 1 -- 172.21.15.163:0/4150400350 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f046c153910 con 0x7f044c0762d0 2024-05-08T06:37:48.107 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.107+0000 7f04697fa640 1 -- 172.21.15.163:0/4150400350 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f046c153910 con 0x7f044c0762d0 2024-05-08T06:37:48.107 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 1 to deep-scrub 2024-05-08T06:37:48.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.109+0000 7f04730a0640 1 -- 172.21.15.163:0/4150400350 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f044c0762d0 msgr2=0x7f044c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.109+0000 7f04730a0640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f044c0762d0 0x7f044c078790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0464007920 tx=0x7f0464008040 comp rx=0 tx=0).stop 2024-05-08T06:37:48.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.109+0000 7f04730a0640 1 -- 172.21.15.163:0/4150400350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f046c0b7140 msgr2=0x7f046c160b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.109+0000 7f04730a0640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f046c0b7140 0x7f046c160b50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f045c007f20 tx=0x7f045c00e570 comp rx=0 tx=0).stop 2024-05-08T06:37:48.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.109+0000 7f04730a0640 1 -- 172.21.15.163:0/4150400350 shutdown_connections 2024-05-08T06:37:48.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.109+0000 7f04730a0640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f044c0762d0 0x7f044c078790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.109+0000 7f04730a0640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046c15f500 0x7f046c1f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.109 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.109+0000 7f04730a0640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f046c0b7140 0x7f046c160b50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.109+0000 7f04730a0640 1 --2- 172.21.15.163:0/4150400350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f046c0b3f60 0x7f046c160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.109+0000 7f04730a0640 1 -- 172.21.15.163:0/4150400350 >> 172.21.15.163:0/4150400350 conn(0x7f046c06d610 msgr2=0x7f046c06fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:48.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.109+0000 7f04730a0640 1 -- 172.21.15.163:0/4150400350 shutdown_connections 2024-05-08T06:37:48.110 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.109+0000 7f04730a0640 1 -- 172.21.15.163:0/4150400350 wait complete. 2024-05-08T06:37:48.120 INFO:tasks.ceph:Scrubbing osd.2 2024-05-08T06:37:48.120 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-08T06:37:48.225 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.225+0000 7f2822cd9640 1 Processor -- start 2024-05-08T06:37:48.225 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.226+0000 7f2822cd9640 1 -- start start 2024-05-08T06:37:48.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.226+0000 7f2822cd9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f281c15c7d0 0x7f281c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.226+0000 7f2822cd9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f281c0b4f80 con 0x7f281c0b49f0 2024-05-08T06:37:48.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.226+0000 7f2822cd9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f281c0b5260 con 0x7f281c06f010 2024-05-08T06:37:48.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.226+0000 7f2822cd9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f281c15d070 con 0x7f281c15c7d0 2024-05-08T06:37:48.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.226+0000 7f2820a75640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f281c15c7d0 0x7f281c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.227+0000 7f2821276640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f281c0b49f0 0x7f281c15db50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:41672/0 (socket says 172.21.15.163:41672) 2024-05-08T06:37:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.227+0000 7f2821276640 1 -- 172.21.15.163:0/2247108763 learned_addr learned my addr 172.21.15.163:0/2247108763 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.227+0000 7f2820a75640 1 -- 172.21.15.163:0/2247108763 >> v1:172.21.15.192:6789/0 conn(0x7f281c06f010 legacy=0x7f281c0b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.227+0000 7f2820a75640 1 -- 172.21.15.163:0/2247108763 >> v1:172.21.15.163:6789/0 conn(0x7f281c0b49f0 legacy=0x7f281c15db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.227+0000 7f281b7fe640 1 -- 172.21.15.163:0/2247108763 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2290049963 0 0) 0x7f281c0b4f80 con 0x7f281c0b49f0 2024-05-08T06:37:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.227+0000 7f2820a75640 1 -- 172.21.15.163:0/2247108763 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c15d350 con 0x7f281c15c7d0 2024-05-08T06:37:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.227+0000 7f2820a75640 1 --2- 172.21.15.163:0/2247108763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f281c15c7d0 0x7f281c06ead0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f280c0097a0 tx=0x7f280c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c4675513a8124e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.228+0000 7f281b7fe640 1 -- 172.21.15.163:0/2247108763 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c037070 con 0x7f281c15c7d0 2024-05-08T06:37:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.228+0000 7f281b7fe640 1 -- 172.21.15.163:0/2247108763 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f280c032440 con 0x7f281c15c7d0 2024-05-08T06:37:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.228+0000 7f281b7fe640 1 -- 172.21.15.163:0/2247108763 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c032810 con 0x7f281c15c7d0 2024-05-08T06:37:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.228+0000 7f2822cd9640 1 -- 172.21.15.163:0/2247108763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f281c15c7d0 msgr2=0x7f281c06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.228+0000 7f2822cd9640 1 --2- 172.21.15.163:0/2247108763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f281c15c7d0 0x7f281c06ead0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f280c0097a0 tx=0x7f280c02a230 comp rx=0 tx=0).stop 2024-05-08T06:37:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.228+0000 7f2822cd9640 1 -- 172.21.15.163:0/2247108763 shutdown_connections 2024-05-08T06:37:48.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.228+0000 7f2822cd9640 1 --2- 172.21.15.163:0/2247108763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f281c15c7d0 0x7f281c06ead0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.228+0000 7f2822cd9640 1 -- 172.21.15.163:0/2247108763 >> 172.21.15.163:0/2247108763 conn(0x7f281c0b2b40 msgr2=0x7f281c0b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:48.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.228+0000 7f2822cd9640 1 -- 172.21.15.163:0/2247108763 shutdown_connections 2024-05-08T06:37:48.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.228+0000 7f2822cd9640 1 -- 172.21.15.163:0/2247108763 wait complete. 2024-05-08T06:37:48.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.229+0000 7f2822cd9640 1 Processor -- start 2024-05-08T06:37:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.229+0000 7f2822cd9640 1 -- start start 2024-05-08T06:37:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.230+0000 7f2822cd9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f281c06f010 0x7f281c150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.230+0000 7f2822cd9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f281c0b49f0 0x7f281c1f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.230+0000 7f2822cd9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f281c15c7d0 0x7f281c1f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.230+0000 7f2822cd9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f281c15cd30 con 0x7f281c06f010 2024-05-08T06:37:48.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.230+0000 7f2822cd9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f281c15cbb0 con 0x7f281c15c7d0 2024-05-08T06:37:48.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.230+0000 7f2822cd9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f281c06e240 con 0x7f281c0b49f0 2024-05-08T06:37:48.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.230+0000 7f2820a75640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f281c06f010 0x7f281c150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.230+0000 7f2820a75640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f281c06f010 0x7f281c150c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34014/0 (socket says 172.21.15.163:34014) 2024-05-08T06:37:48.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.230+0000 7f2820a75640 1 -- 172.21.15.163:0/2641158328 learned_addr learned my addr 172.21.15.163:0/2641158328 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:48.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.231+0000 7f2820a75640 1 -- 172.21.15.163:0/2641158328 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f281c0b49f0 msgr2=0x7f281c1f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.231+0000 7f2820a75640 1 --2- 172.21.15.163:0/2641158328 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f281c0b49f0 0x7f281c1f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.231+0000 7f2820a75640 1 -- 172.21.15.163:0/2641158328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f281c15c7d0 msgr2=0x7f281c1f5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.231+0000 7f2820a75640 1 --2- 172.21.15.163:0/2641158328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f281c15c7d0 0x7f281c1f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.231+0000 7f2820a75640 1 -- 172.21.15.163:0/2641158328 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c14eef0 con 0x7f281c06f010 2024-05-08T06:37:48.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.231+0000 7f2820a75640 1 --2- 172.21.15.163:0/2641158328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f281c06f010 0x7f281c150c40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f280c0097a0 tx=0x7f280c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:48.238 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.231+0000 7f28197fa640 1 -- 172.21.15.163:0/2641158328 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c037050 con 0x7f281c06f010 2024-05-08T06:37:48.238 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.231+0000 7f28197fa640 1 -- 172.21.15.163:0/2641158328 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f280c038040 con 0x7f281c06f010 2024-05-08T06:37:48.238 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.231+0000 7f2822cd9640 1 -- 172.21.15.163:0/2641158328 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281c14f1d0 con 0x7f281c06f010 2024-05-08T06:37:48.238 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.231+0000 7f28197fa640 1 -- 172.21.15.163:0/2641158328 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c004470 con 0x7f281c06f010 2024-05-08T06:37:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.231+0000 7f2822cd9640 1 -- 172.21.15.163:0/2641158328 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281c14f790 con 0x7f281c06f010 2024-05-08T06:37:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.232+0000 7f2822cd9640 1 -- 172.21.15.163:0/2641158328 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f281c14eca0 con 0x7f281c06f010 2024-05-08T06:37:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.233+0000 7f28197fa640 1 -- 172.21.15.163:0/2641158328 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f280c030900 con 0x7f281c06f010 2024-05-08T06:37:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.233+0000 7f28197fa640 1 --2- 172.21.15.163:0/2641158328 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f28080762d0 0x7f2808078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.233+0000 7f28197fa640 1 -- 172.21.15.163:0/2641158328 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f280c0b7f90 con 0x7f281c06f010 2024-05-08T06:37:48.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.233+0000 7f281bfff640 1 --2- 172.21.15.163:0/2641158328 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f28080762d0 0x7f2808078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.233+0000 7f28197fa640 1 --2- 172.21.15.163:0/2641158328 >> [v2:172.21.15.163:6808/601639604,v1:172.21.15.163:6809/601639604] conn(0x7f280807c230 0x7f280807e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.233+0000 7f28197fa640 1 -- 172.21.15.163:0/2641158328 --> [v2:172.21.15.163:6808/601639604,v1:172.21.15.163:6809/601639604] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f280c02cea0 con 0x7f280807c230 2024-05-08T06:37:48.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.233+0000 7f28197fa640 1 -- 172.21.15.163:0/2641158328 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f280c0bf050 con 0x7f281c06f010 2024-05-08T06:37:48.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.233+0000 7f2821276640 1 --2- 172.21.15.163:0/2641158328 >> [v2:172.21.15.163:6808/601639604,v1:172.21.15.163:6809/601639604] conn(0x7f280807c230 0x7f280807e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.240 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.234+0000 7f281bfff640 1 --2- 172.21.15.163:0/2641158328 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f28080762d0 0x7f2808078790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f281c1500c0 tx=0x7f2810007970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:48.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.234+0000 7f2821276640 1 --2- 172.21.15.163:0/2641158328 >> [v2:172.21.15.163:6808/601639604,v1:172.21.15.163:6809/601639604] conn(0x7f280807c230 0x7f280807e650 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:48.241 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.234+0000 7f28197fa640 1 -- 172.21.15.163:0/2641158328 <== osd.2 v2:172.21.15.163:6808/601639604 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f280c02cea0 con 0x7f280807c230 2024-05-08T06:37:48.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.247+0000 7f2822cd9640 1 -- 172.21.15.163:0/2641158328 --> [v2:172.21.15.163:6808/601639604,v1:172.21.15.163:6809/601639604] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f281c06a550 con 0x7f280807c230 2024-05-08T06:37:48.246 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.247+0000 7f28197fa640 1 -- 172.21.15.163:0/2641158328 <== osd.2 v2:172.21.15.163:6808/601639604 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f281c06a550 con 0x7f280807c230 2024-05-08T06:37:48.247 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:37:48.247 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T06:37:48.247 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:37:48.248 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.248+0000 7f2822cd9640 1 -- 172.21.15.163:0/2641158328 >> [v2:172.21.15.163:6808/601639604,v1:172.21.15.163:6809/601639604] conn(0x7f280807c230 msgr2=0x7f280807e650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.248 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.248+0000 7f2822cd9640 1 --2- 172.21.15.163:0/2641158328 >> [v2:172.21.15.163:6808/601639604,v1:172.21.15.163:6809/601639604] conn(0x7f280807c230 0x7f280807e650 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.248 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.248+0000 7f2822cd9640 1 -- 172.21.15.163:0/2641158328 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f28080762d0 msgr2=0x7f2808078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.248 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.248+0000 7f2822cd9640 1 --2- 172.21.15.163:0/2641158328 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f28080762d0 0x7f2808078790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f281c1500c0 tx=0x7f2810007970 comp rx=0 tx=0).stop 2024-05-08T06:37:48.248 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.248+0000 7f2822cd9640 1 -- 172.21.15.163:0/2641158328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f281c06f010 msgr2=0x7f281c150c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.249 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.248+0000 7f2822cd9640 1 --2- 172.21.15.163:0/2641158328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f281c06f010 0x7f281c150c40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f280c0097a0 tx=0x7f280c0046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.249 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.248+0000 7f2821276640 1 -- 172.21.15.163:0/2641158328 reap_dead start 2024-05-08T06:37:48.249 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.248+0000 7f2822cd9640 1 -- 172.21.15.163:0/2641158328 shutdown_connections 2024-05-08T06:37:48.249 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.248+0000 7f2822cd9640 1 -- 172.21.15.163:0/2641158328 >> 172.21.15.163:0/2641158328 conn(0x7f281c0b2b40 msgr2=0x7f281c0b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:48.249 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.248+0000 7f2822cd9640 1 -- 172.21.15.163:0/2641158328 shutdown_connections 2024-05-08T06:37:48.249 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.248+0000 7f2822cd9640 1 -- 172.21.15.163:0/2641158328 wait complete. 2024-05-08T06:37:48.258 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-08T06:37:48.359 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.360+0000 7f18dedf2640 1 Processor -- start 2024-05-08T06:37:48.360 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.360+0000 7f18dedf2640 1 -- start start 2024-05-08T06:37:48.361 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.361+0000 7f18dedf2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d80d3d10 0x7f18d81629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.361 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.361+0000 7f18dedf2640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f18d8162f00 0x7f18d806d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.361 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.361+0000 7f18dedf2640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18d80d1560 con 0x7f18d80d3930 2024-05-08T06:37:48.361 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.361+0000 7f18dedf2640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f18d8163390 con 0x7f18d8162f00 2024-05-08T06:37:48.361 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.361+0000 7f18dedf2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f18d80d0cc0 con 0x7f18d80d3d10 2024-05-08T06:37:48.361 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.361+0000 7f18cffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d80d3d10 0x7f18d81629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.361+0000 7f18dd38f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f18d8162f00 0x7f18d806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.361+0000 7f18dd38f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f18d8162f00 0x7f18d806d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.163:41730/0 (socket says 172.21.15.163:41730) 2024-05-08T06:37:48.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.361+0000 7f18dd38f640 1 -- 172.21.15.163:0/2643570048 learned_addr learned my addr 172.21.15.163:0/2643570048 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:48.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.362+0000 7f18dd38f640 1 -- 172.21.15.163:0/2643570048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d80d3d10 msgr2=0x7f18d81629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:48.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.362+0000 7f18dd38f640 1 --2- 172.21.15.163:0/2643570048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d80d3d10 0x7f18d81629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.362+0000 7f18dd38f640 1 -- 172.21.15.163:0/2643570048 >> v1:172.21.15.192:6789/0 conn(0x7f18d80d3930 legacy=0x7f18d80d59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.362+0000 7f18dd38f640 1 -- 172.21.15.163:0/2643570048 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d80d0fa0 con 0x7f18d8162f00 2024-05-08T06:37:48.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.362+0000 7f18cffff640 1 --2- 172.21.15.163:0/2643570048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d80d3d10 0x7f18d81629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:37:48.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.362+0000 7f18dd38f640 1 --2- 172.21.15.163:0/2643570048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18d8162f00 0x7f18d806d960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009870 tx=0x7f18d002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5697f1c4c8dcf8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:48.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.363+0000 7f18cf7fe640 1 -- 172.21.15.163:0/2643570048 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0037070 con 0x7f18d8162f00 2024-05-08T06:37:48.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.363+0000 7f18cf7fe640 1 -- 172.21.15.163:0/2643570048 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d0032440 con 0x7f18d8162f00 2024-05-08T06:37:48.367 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.363+0000 7f18cf7fe640 1 -- 172.21.15.163:0/2643570048 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0032810 con 0x7f18d8162f00 2024-05-08T06:37:48.367 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.363+0000 7f18dedf2640 1 -- 172.21.15.163:0/2643570048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18d8162f00 msgr2=0x7f18d806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.367 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.363+0000 7f18dedf2640 1 --2- 172.21.15.163:0/2643570048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18d8162f00 0x7f18d806d960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009870 tx=0x7f18d002a270 comp rx=0 tx=0).stop 2024-05-08T06:37:48.367 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.363+0000 7f18dedf2640 1 -- 172.21.15.163:0/2643570048 shutdown_connections 2024-05-08T06:37:48.367 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.363+0000 7f18dedf2640 1 --2- 172.21.15.163:0/2643570048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18d8162f00 0x7f18d806d960 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.368 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.363+0000 7f18dedf2640 1 --2- 172.21.15.163:0/2643570048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d80d3d10 0x7f18d81629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.368 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.363+0000 7f18dedf2640 1 -- 172.21.15.163:0/2643570048 >> 172.21.15.163:0/2643570048 conn(0x7f18d808cfa0 msgr2=0x7f18d80d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:48.368 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.363+0000 7f18dedf2640 1 -- 172.21.15.163:0/2643570048 shutdown_connections 2024-05-08T06:37:48.368 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.363+0000 7f18dedf2640 1 -- 172.21.15.163:0/2643570048 wait complete. 2024-05-08T06:37:48.368 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.364+0000 7f18dedf2640 1 Processor -- start 2024-05-08T06:37:48.368 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.364+0000 7f18dedf2640 1 -- start start 2024-05-08T06:37:48.368 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.365+0000 7f18dedf2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d80d3930 0x7f18d81f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.369 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.365+0000 7f18dedf2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d80d3d10 0x7f18d8158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.369 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.365+0000 7f18dedf2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18d8162f00 0x7f18d80d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.369 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.365+0000 7f18dedf2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f18d80d0970 con 0x7f18d80d3930 2024-05-08T06:37:48.369 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.365+0000 7f18dedf2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f18d80d60b0 con 0x7f18d8162f00 2024-05-08T06:37:48.369 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.365+0000 7f18dedf2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f18d806dea0 con 0x7f18d80d3d10 2024-05-08T06:37:48.369 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.365+0000 7f18dcb8e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d80d3930 0x7f18d81f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.369 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.365+0000 7f18dcb8e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d80d3930 0x7f18d81f4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34018/0 (socket says 172.21.15.163:34018) 2024-05-08T06:37:48.370 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.365+0000 7f18dcb8e640 1 -- 172.21.15.163:0/2905478404 learned_addr learned my addr 172.21.15.163:0/2905478404 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:48.370 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.366+0000 7f18dd38f640 1 --2- 172.21.15.163:0/2905478404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18d8162f00 0x7f18d80d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.370 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.366+0000 7f18cffff640 1 --2- 172.21.15.163:0/2905478404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d80d3d10 0x7f18d8158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.370 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.366+0000 7f18dcb8e640 1 -- 172.21.15.163:0/2905478404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d80d3d10 msgr2=0x7f18d8158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.370 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.366+0000 7f18dcb8e640 1 --2- 172.21.15.163:0/2905478404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d80d3d10 0x7f18d8158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.370 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.366+0000 7f18dcb8e640 1 -- 172.21.15.163:0/2905478404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18d8162f00 msgr2=0x7f18d80d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.371 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.366+0000 7f18dcb8e640 1 --2- 172.21.15.163:0/2905478404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18d8162f00 0x7f18d80d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.371 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.366+0000 7f18dcb8e640 1 -- 172.21.15.163:0/2905478404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d80cdbc0 con 0x7f18d80d3930 2024-05-08T06:37:48.371 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.366+0000 7f18dcb8e640 1 --2- 172.21.15.163:0/2905478404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d80d3930 0x7f18d81f4a80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f18d400c870 tx=0x7f18d400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:48.371 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.366+0000 7f18cd7fa640 1 -- 172.21.15.163:0/2905478404 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4003b70 con 0x7f18d80d3930 2024-05-08T06:37:48.371 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.366+0000 7f18cd7fa640 1 -- 172.21.15.163:0/2905478404 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d4003d10 con 0x7f18d80d3930 2024-05-08T06:37:48.371 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.366+0000 7f18dedf2640 1 -- 172.21.15.163:0/2905478404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d80cdea0 con 0x7f18d80d3930 2024-05-08T06:37:48.372 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.366+0000 7f18cd7fa640 1 -- 172.21.15.163:0/2905478404 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d400aa90 con 0x7f18d80d3930 2024-05-08T06:37:48.372 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.366+0000 7f18dedf2640 1 -- 172.21.15.163:0/2905478404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d80ce460 con 0x7f18d80d3930 2024-05-08T06:37:48.372 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.367+0000 7f18dedf2640 1 -- 172.21.15.163:0/2905478404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d806a4b0 con 0x7f18d80d3930 2024-05-08T06:37:48.373 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.368+0000 7f18cd7fa640 1 -- 172.21.15.163:0/2905478404 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f18d400ac30 con 0x7f18d80d3930 2024-05-08T06:37:48.373 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.368+0000 7f18cd7fa640 1 --2- 172.21.15.163:0/2905478404 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f18c40762d0 0x7f18c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.373 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.368+0000 7f18cffff640 1 --2- 172.21.15.163:0/2905478404 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f18c40762d0 0x7f18c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.368+0000 7f18cd7fa640 1 -- 172.21.15.163:0/2905478404 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f18d4097160 con 0x7f18d80d3930 2024-05-08T06:37:48.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.369+0000 7f18cffff640 1 --2- 172.21.15.163:0/2905478404 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f18c40762d0 0x7f18c4078790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009560 tx=0x7f18c8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:48.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.373+0000 7f18cd7fa640 1 -- 172.21.15.163:0/2905478404 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f18d4060710 con 0x7f18d80d3930 2024-05-08T06:37:48.553 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.554+0000 7f18dedf2640 1 -- 172.21.15.163:0/2905478404 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f18d80cd970 con 0x7f18c40762d0 2024-05-08T06:37:48.554 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.554+0000 7f18cd7fa640 1 -- 172.21.15.163:0/2905478404 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f18d80cd970 con 0x7f18c40762d0 2024-05-08T06:37:48.554 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 2 to deep-scrub 2024-05-08T06:37:48.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.556+0000 7f18dedf2640 1 -- 172.21.15.163:0/2905478404 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f18c40762d0 msgr2=0x7f18c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.556+0000 7f18dedf2640 1 --2- 172.21.15.163:0/2905478404 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f18c40762d0 0x7f18c4078790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009560 tx=0x7f18c8009190 comp rx=0 tx=0).stop 2024-05-08T06:37:48.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.556+0000 7f18dedf2640 1 -- 172.21.15.163:0/2905478404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d80d3930 msgr2=0x7f18d81f4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.556+0000 7f18dedf2640 1 --2- 172.21.15.163:0/2905478404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d80d3930 0x7f18d81f4a80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f18d400c870 tx=0x7f18d400cd30 comp rx=0 tx=0).stop 2024-05-08T06:37:48.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.556+0000 7f18dedf2640 1 -- 172.21.15.163:0/2905478404 shutdown_connections 2024-05-08T06:37:48.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.556+0000 7f18dedf2640 1 --2- 172.21.15.163:0/2905478404 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f18c40762d0 0x7f18c4078790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.556+0000 7f18dedf2640 1 --2- 172.21.15.163:0/2905478404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18d8162f00 0x7f18d80d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.556+0000 7f18dedf2640 1 --2- 172.21.15.163:0/2905478404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d80d3d10 0x7f18d8158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.556+0000 7f18dedf2640 1 --2- 172.21.15.163:0/2905478404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d80d3930 0x7f18d81f4a80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.556+0000 7f18dedf2640 1 -- 172.21.15.163:0/2905478404 >> 172.21.15.163:0/2905478404 conn(0x7f18d808cfa0 msgr2=0x7f18d80b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:48.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.557+0000 7f18dedf2640 1 -- 172.21.15.163:0/2905478404 shutdown_connections 2024-05-08T06:37:48.557 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.557+0000 7f18dedf2640 1 -- 172.21.15.163:0/2905478404 wait complete. 2024-05-08T06:37:48.567 INFO:tasks.ceph:Scrubbing osd.3 2024-05-08T06:37:48.567 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-08T06:37:48.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.672+0000 7ff1bbf04640 1 Processor -- start 2024-05-08T06:37:48.671 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.672+0000 7ff1bbf04640 1 -- start start 2024-05-08T06:37:48.680 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1bbf04640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff1b415c120 0x7ff1b406fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1bbf04640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1b40b0390 0x7ff1b40b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1bbf04640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff1b40b59f0 0x7ff1b415fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1bbf04640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7ff1b415c500 con 0x7ff1b40b59f0 2024-05-08T06:37:48.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1bbf04640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff1b40afe90 con 0x7ff1b415c120 2024-05-08T06:37:48.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1bbf04640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b40b49d0 con 0x7ff1b40b0390 2024-05-08T06:37:48.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1ba4a1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff1b40b59f0 0x7ff1b415fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1ba4a1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff1b40b59f0 0x7ff1b415fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34024/0 (socket says 172.21.15.163:34024) 2024-05-08T06:37:48.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1ba4a1640 1 -- 172.21.15.163:0/3676680841 learned_addr learned my addr 172.21.15.163:0/3676680841 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:48.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1b9ca0640 1 --2- 172.21.15.163:0/3676680841 >> v2:172.21.15.192:3300/0 conn(0x7ff1b415c120 0x7ff1b406fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1b949f640 1 --2- 172.21.15.163:0/3676680841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1b40b0390 0x7ff1b40b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1ba4a1640 1 -- 172.21.15.163:0/3676680841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1b40b0390 msgr2=0x7ff1b40b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:48.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1ba4a1640 1 --2- 172.21.15.163:0/3676680841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1b40b0390 0x7ff1b40b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1ba4a1640 1 -- 172.21.15.163:0/3676680841 >> v2:172.21.15.192:3300/0 conn(0x7ff1b415c120 msgr2=0x7ff1b406fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:48.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1ba4a1640 1 --2- 172.21.15.163:0/3676680841 >> v2:172.21.15.192:3300/0 conn(0x7ff1b415c120 0x7ff1b406fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.673+0000 7ff1ba4a1640 1 -- 172.21.15.163:0/3676680841 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b40b4cb0 con 0x7ff1b40b59f0 2024-05-08T06:37:48.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.674+0000 7ff1ba4a1640 1 --2- 172.21.15.163:0/3676680841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1b40b59f0 0x7ff1b415fc60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4070450 tx=0x7ff1ac02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcd4335f2bc6a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:48.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.674+0000 7ff1b8c9e640 1 -- 172.21.15.163:0/3676680841 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac037070 con 0x7ff1b40b59f0 2024-05-08T06:37:48.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.674+0000 7ff1b8c9e640 1 -- 172.21.15.163:0/3676680841 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ac032440 con 0x7ff1b40b59f0 2024-05-08T06:37:48.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.674+0000 7ff1b8c9e640 1 -- 172.21.15.163:0/3676680841 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac032810 con 0x7ff1b40b59f0 2024-05-08T06:37:48.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.675+0000 7ff1bbf04640 1 -- 172.21.15.163:0/3676680841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1b40b59f0 msgr2=0x7ff1b415fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.675+0000 7ff1bbf04640 1 --2- 172.21.15.163:0/3676680841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1b40b59f0 0x7ff1b415fc60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4070450 tx=0x7ff1ac02a140 comp rx=0 tx=0).stop 2024-05-08T06:37:48.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.675+0000 7ff1bbf04640 1 -- 172.21.15.163:0/3676680841 shutdown_connections 2024-05-08T06:37:48.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.675+0000 7ff1bbf04640 1 --2- 172.21.15.163:0/3676680841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1b40b59f0 0x7ff1b415fc60 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.675+0000 7ff1bbf04640 1 --2- 172.21.15.163:0/3676680841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1b40b0390 0x7ff1b40b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.675+0000 7ff1bbf04640 1 --2- 172.21.15.163:0/3676680841 >> v2:172.21.15.192:3300/0 conn(0x7ff1b415c120 0x7ff1b406fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.684 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.675+0000 7ff1bbf04640 1 -- 172.21.15.163:0/3676680841 >> 172.21.15.163:0/3676680841 conn(0x7ff1b40b2b40 msgr2=0x7ff1b40b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:48.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.675+0000 7ff1bbf04640 1 -- 172.21.15.163:0/3676680841 shutdown_connections 2024-05-08T06:37:48.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.675+0000 7ff1bbf04640 1 -- 172.21.15.163:0/3676680841 wait complete. 2024-05-08T06:37:48.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.676+0000 7ff1bbf04640 1 Processor -- start 2024-05-08T06:37:48.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.676+0000 7ff1bbf04640 1 -- start start 2024-05-08T06:37:48.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1bbf04640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1b40b0390 0x7ff1b4140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1bbf04640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1b40b59f0 0x7ff1b413def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1bbf04640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1b415c120 0x7ff1b415c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1bbf04640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4160320 con 0x7ff1b415c120 2024-05-08T06:37:48.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1bbf04640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41601a0 con 0x7ff1b40b59f0 2024-05-08T06:37:48.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1bbf04640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b41604a0 con 0x7ff1b40b0390 2024-05-08T06:37:48.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1ba4a1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1b415c120 0x7ff1b415c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1ba4a1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1b415c120 0x7ff1b415c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34026/0 (socket says 172.21.15.163:34026) 2024-05-08T06:37:48.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1ba4a1640 1 -- 172.21.15.163:0/965700727 learned_addr learned my addr 172.21.15.163:0/965700727 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:48.686 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1b9ca0640 1 --2- 172.21.15.163:0/965700727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1b40b0390 0x7ff1b4140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1b949f640 1 --2- 172.21.15.163:0/965700727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1b40b59f0 0x7ff1b413def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1ba4a1640 1 -- 172.21.15.163:0/965700727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1b40b0390 msgr2=0x7ff1b4140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1ba4a1640 1 --2- 172.21.15.163:0/965700727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1b40b0390 0x7ff1b4140bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1ba4a1640 1 -- 172.21.15.163:0/965700727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1b40b59f0 msgr2=0x7ff1b413def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1ba4a1640 1 --2- 172.21.15.163:0/965700727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1b40b59f0 0x7ff1b413def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1ba4a1640 1 -- 172.21.15.163:0/965700727 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b41545a0 con 0x7ff1b415c120 2024-05-08T06:37:48.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.677+0000 7ff1ba4a1640 1 --2- 172.21.15.163:0/965700727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1b415c120 0x7ff1b415c6e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0090b0 tx=0x7ff1ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:48.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.678+0000 7ff1aa7fc640 1 -- 172.21.15.163:0/965700727 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac039800 con 0x7ff1b415c120 2024-05-08T06:37:48.687 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.678+0000 7ff1aa7fc640 1 -- 172.21.15.163:0/965700727 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ac0399c0 con 0x7ff1b415c120 2024-05-08T06:37:48.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.678+0000 7ff1bbf04640 1 -- 172.21.15.163:0/965700727 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b4154880 con 0x7ff1b415c120 2024-05-08T06:37:48.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.678+0000 7ff1aa7fc640 1 -- 172.21.15.163:0/965700727 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac039db0 con 0x7ff1b415c120 2024-05-08T06:37:48.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.678+0000 7ff1bbf04640 1 -- 172.21.15.163:0/965700727 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b4154e40 con 0x7ff1b415c120 2024-05-08T06:37:48.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.678+0000 7ff1bbf04640 1 -- 172.21.15.163:0/965700727 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff1b406a550 con 0x7ff1b415c120 2024-05-08T06:37:48.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.679+0000 7ff1aa7fc640 1 -- 172.21.15.163:0/965700727 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff1ac04d3f0 con 0x7ff1b415c120 2024-05-08T06:37:48.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.680+0000 7ff1aa7fc640 1 --2- 172.21.15.163:0/965700727 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff1980762d0 0x7ff198078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.680+0000 7ff1b9ca0640 1 --2- 172.21.15.163:0/965700727 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff1980762d0 0x7ff198078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.688 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.680+0000 7ff1aa7fc640 1 -- 172.21.15.163:0/965700727 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7ff1ac0b9d20 con 0x7ff1b415c120 2024-05-08T06:37:48.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.680+0000 7ff1aa7fc640 1 --2- 172.21.15.163:0/965700727 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x7ff19807c230 0x7ff19807e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.680+0000 7ff1aa7fc640 1 -- 172.21.15.163:0/965700727 --> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1ac032440 con 0x7ff19807c230 2024-05-08T06:37:48.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.680+0000 7ff1aa7fc640 1 -- 172.21.15.163:0/965700727 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff1ac0ba1b0 con 0x7ff1b415c120 2024-05-08T06:37:48.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.680+0000 7ff1b949f640 1 --2- 172.21.15.163:0/965700727 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x7ff19807c230 0x7ff19807e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.680+0000 7ff1b9ca0640 1 --2- 172.21.15.163:0/965700727 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff1980762d0 0x7ff198078790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0011340 tx=0x7ff1b000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:48.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.680+0000 7ff1b949f640 1 --2- 172.21.15.163:0/965700727 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x7ff19807c230 0x7ff19807e650 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:48.689 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.681+0000 7ff1aa7fc640 1 -- 172.21.15.163:0/965700727 <== osd.3 v2:172.21.15.163:6816/127592521 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1ac032440 con 0x7ff19807c230 2024-05-08T06:37:48.693 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.694+0000 7ff1bbf04640 1 -- 172.21.15.163:0/965700727 --> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff1b4155260 con 0x7ff19807c230 2024-05-08T06:37:48.693 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.694+0000 7ff1aa7fc640 1 -- 172.21.15.163:0/965700727 <== osd.3 v2:172.21.15.163:6816/127592521 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff1b4155260 con 0x7ff19807c230 2024-05-08T06:37:48.693 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:37:48.694 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T06:37:48.694 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:37:48.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.694+0000 7ff1bbf04640 1 -- 172.21.15.163:0/965700727 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x7ff19807c230 msgr2=0x7ff19807e650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.694+0000 7ff1bbf04640 1 --2- 172.21.15.163:0/965700727 >> [v2:172.21.15.163:6816/127592521,v1:172.21.15.163:6817/127592521] conn(0x7ff19807c230 0x7ff19807e650 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.694+0000 7ff1bbf04640 1 -- 172.21.15.163:0/965700727 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff1980762d0 msgr2=0x7ff198078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.694+0000 7ff1bbf04640 1 --2- 172.21.15.163:0/965700727 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7ff1980762d0 0x7ff198078790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0011340 tx=0x7ff1b000b040 comp rx=0 tx=0).stop 2024-05-08T06:37:48.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.694+0000 7ff1bbf04640 1 -- 172.21.15.163:0/965700727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1b415c120 msgr2=0x7ff1b415c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.694+0000 7ff1bbf04640 1 --2- 172.21.15.163:0/965700727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1b415c120 0x7ff1b415c6e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0090b0 tx=0x7ff1ac0046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.695+0000 7ff1ba4a1640 1 -- 172.21.15.163:0/965700727 reap_dead start 2024-05-08T06:37:48.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.695+0000 7ff1bbf04640 1 -- 172.21.15.163:0/965700727 shutdown_connections 2024-05-08T06:37:48.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.695+0000 7ff1bbf04640 1 -- 172.21.15.163:0/965700727 >> 172.21.15.163:0/965700727 conn(0x7ff1b40b2b40 msgr2=0x7ff1b40b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:48.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.695+0000 7ff1bbf04640 1 -- 172.21.15.163:0/965700727 shutdown_connections 2024-05-08T06:37:48.696 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.695+0000 7ff1bbf04640 1 -- 172.21.15.163:0/965700727 wait complete. 2024-05-08T06:37:48.704 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-08T06:37:48.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.807+0000 7fed77bf7640 1 Processor -- start 2024-05-08T06:37:48.806 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.807+0000 7fed77bf7640 1 -- start start 2024-05-08T06:37:48.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed77bf7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fed700d1490 0x7fed700b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed77bf7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed700b1610 0x7fed7006c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.810 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed77bf7640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fed7006c860 0x7fed70160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.810 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed77bf7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fed70074330 con 0x7fed700d1490 2024-05-08T06:37:48.810 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed77bf7640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fed70073480 con 0x7fed7006c860 2024-05-08T06:37:48.810 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed77bf7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fed70073720 con 0x7fed700b1610 2024-05-08T06:37:48.810 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed75993640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fed700d1490 0x7fed700b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.811 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed75993640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fed700d1490 0x7fed700b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34042/0 (socket says 172.21.15.163:34042) 2024-05-08T06:37:48.811 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed75993640 1 -- 172.21.15.163:0/1018168973 learned_addr learned my addr 172.21.15.163:0/1018168973 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:48.811 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed75192640 1 --2- 172.21.15.163:0/1018168973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed700b1610 0x7fed7006c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.811 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed76194640 1 --2- 172.21.15.163:0/1018168973 >> v2:172.21.15.192:3300/0 conn(0x7fed7006c860 0x7fed70160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.811 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed75993640 1 -- 172.21.15.163:0/1018168973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed700b1610 msgr2=0x7fed7006c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:48.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed75993640 1 --2- 172.21.15.163:0/1018168973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed700b1610 0x7fed7006c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed75993640 1 -- 172.21.15.163:0/1018168973 >> v2:172.21.15.192:3300/0 conn(0x7fed7006c860 msgr2=0x7fed70160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:48.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed75993640 1 --2- 172.21.15.163:0/1018168973 >> v2:172.21.15.192:3300/0 conn(0x7fed7006c860 0x7fed70160fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.808+0000 7fed75993640 1 -- 172.21.15.163:0/1018168973 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed70073a00 con 0x7fed700d1490 2024-05-08T06:37:48.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.809+0000 7fed75993640 1 --2- 172.21.15.163:0/1018168973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed700d1490 0x7fed700b10d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fed70160f00 tx=0x7fed6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d49090436d2531 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:48.812 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.809+0000 7fed74991640 1 -- 172.21.15.163:0/1018168973 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c037070 con 0x7fed700d1490 2024-05-08T06:37:48.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.809+0000 7fed74991640 1 -- 172.21.15.163:0/1018168973 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed6c032440 con 0x7fed700d1490 2024-05-08T06:37:48.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.809+0000 7fed74991640 1 -- 172.21.15.163:0/1018168973 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c032810 con 0x7fed700d1490 2024-05-08T06:37:48.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.810+0000 7fed77bf7640 1 -- 172.21.15.163:0/1018168973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed700d1490 msgr2=0x7fed700b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.810+0000 7fed77bf7640 1 --2- 172.21.15.163:0/1018168973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed700d1490 0x7fed700b10d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fed70160f00 tx=0x7fed6c02a1e0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.810+0000 7fed77bf7640 1 -- 172.21.15.163:0/1018168973 shutdown_connections 2024-05-08T06:37:48.813 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.810+0000 7fed77bf7640 1 --2- 172.21.15.163:0/1018168973 >> v2:172.21.15.192:3300/0 conn(0x7fed7006c860 0x7fed70160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.810+0000 7fed77bf7640 1 --2- 172.21.15.163:0/1018168973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed700b1610 0x7fed7006c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.810+0000 7fed77bf7640 1 --2- 172.21.15.163:0/1018168973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed700d1490 0x7fed700b10d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.810+0000 7fed77bf7640 1 -- 172.21.15.163:0/1018168973 >> 172.21.15.163:0/1018168973 conn(0x7fed700b3c30 msgr2=0x7fed700b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:48.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.810+0000 7fed77bf7640 1 -- 172.21.15.163:0/1018168973 shutdown_connections 2024-05-08T06:37:48.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.810+0000 7fed77bf7640 1 -- 172.21.15.163:0/1018168973 wait complete. 2024-05-08T06:37:48.818 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.811+0000 7fed77bf7640 1 Processor -- start 2024-05-08T06:37:48.818 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.811+0000 7fed77bf7640 1 -- start start 2024-05-08T06:37:48.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed77bf7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed7006c860 0x7fed7013f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed77bf7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed700b1610 0x7fed700b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed77bf7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed700d1490 0x7fed7013e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed77bf7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fed701614e0 con 0x7fed700d1490 2024-05-08T06:37:48.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed77bf7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fed70073fc0 con 0x7fed7006c860 2024-05-08T06:37:48.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed77bf7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fed70161660 con 0x7fed700b1610 2024-05-08T06:37:48.820 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed76194640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed700d1490 0x7fed7013e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.820 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed76194640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed700d1490 0x7fed7013e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34048/0 (socket says 172.21.15.163:34048) 2024-05-08T06:37:48.820 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed76194640 1 -- 172.21.15.163:0/674689065 learned_addr learned my addr 172.21.15.163:0/674689065 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:48.820 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed75192640 1 --2- 172.21.15.163:0/674689065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed700b1610 0x7fed700b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.820 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed75993640 1 --2- 172.21.15.163:0/674689065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed7006c860 0x7fed7013f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.820 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed76194640 1 -- 172.21.15.163:0/674689065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed700b1610 msgr2=0x7fed700b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.820 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed76194640 1 --2- 172.21.15.163:0/674689065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed700b1610 0x7fed700b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed76194640 1 -- 172.21.15.163:0/674689065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed7006c860 msgr2=0x7fed7013f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:48.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed76194640 1 --2- 172.21.15.163:0/674689065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed7006c860 0x7fed7013f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:48.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed76194640 1 -- 172.21.15.163:0/674689065 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7013fc70 con 0x7fed700d1490 2024-05-08T06:37:48.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.812+0000 7fed76194640 1 --2- 172.21.15.163:0/674689065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed700d1490 0x7fed7013e700 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fed6800eed0 tx=0x7fed6800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:48.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.813+0000 7fed667fc640 1 -- 172.21.15.163:0/674689065 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68018020 con 0x7fed700d1490 2024-05-08T06:37:48.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.813+0000 7fed667fc640 1 -- 172.21.15.163:0/674689065 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed68014400 con 0x7fed700d1490 2024-05-08T06:37:48.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.813+0000 7fed77bf7640 1 -- 172.21.15.163:0/674689065 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7013ff50 con 0x7fed700d1490 2024-05-08T06:37:48.822 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.813+0000 7fed667fc640 1 -- 172.21.15.163:0/674689065 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed680147f0 con 0x7fed700d1490 2024-05-08T06:37:48.822 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.813+0000 7fed77bf7640 1 -- 172.21.15.163:0/674689065 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed70140510 con 0x7fed700d1490 2024-05-08T06:37:48.822 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.814+0000 7fed77bf7640 1 -- 172.21.15.163:0/674689065 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed7006a4b0 con 0x7fed700d1490 2024-05-08T06:37:48.822 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.814+0000 7fed667fc640 1 -- 172.21.15.163:0/674689065 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fed68005d10 con 0x7fed700d1490 2024-05-08T06:37:48.822 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.815+0000 7fed667fc640 1 --2- 172.21.15.163:0/674689065 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fed540762d0 0x7fed54078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:48.822 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.815+0000 7fed75993640 1 --2- 172.21.15.163:0/674689065 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fed540762d0 0x7fed54078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:48.823 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.815+0000 7fed667fc640 1 -- 172.21.15.163:0/674689065 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fed680977a0 con 0x7fed700d1490 2024-05-08T06:37:48.823 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.815+0000 7fed75993640 1 --2- 172.21.15.163:0/674689065 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fed540762d0 0x7fed54078790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed6c034000 tx=0x7fed6c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:48.823 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.819+0000 7fed667fc640 1 -- 172.21.15.163:0/674689065 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fed68022070 con 0x7fed700d1490 2024-05-08T06:37:48.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.998+0000 7fed77bf7640 1 -- 172.21.15.163:0/674689065 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fed7006d170 con 0x7fed540762d0 2024-05-08T06:37:48.999 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:48.998+0000 7fed667fc640 1 -- 172.21.15.163:0/674689065 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fed7006d170 con 0x7fed540762d0 2024-05-08T06:37:48.999 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 3 to deep-scrub 2024-05-08T06:37:49.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.000+0000 7fed77bf7640 1 -- 172.21.15.163:0/674689065 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fed540762d0 msgr2=0x7fed54078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.000+0000 7fed77bf7640 1 --2- 172.21.15.163:0/674689065 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fed540762d0 0x7fed54078790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed6c034000 tx=0x7fed6c02e000 comp rx=0 tx=0).stop 2024-05-08T06:37:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.000+0000 7fed77bf7640 1 -- 172.21.15.163:0/674689065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed700d1490 msgr2=0x7fed7013e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.000+0000 7fed77bf7640 1 --2- 172.21.15.163:0/674689065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed700d1490 0x7fed7013e700 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fed6800eed0 tx=0x7fed6800b480 comp rx=0 tx=0).stop 2024-05-08T06:37:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.000+0000 7fed77bf7640 1 -- 172.21.15.163:0/674689065 shutdown_connections 2024-05-08T06:37:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.000+0000 7fed77bf7640 1 --2- 172.21.15.163:0/674689065 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fed540762d0 0x7fed54078790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.000+0000 7fed77bf7640 1 --2- 172.21.15.163:0/674689065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed700d1490 0x7fed7013e700 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.000+0000 7fed77bf7640 1 --2- 172.21.15.163:0/674689065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed700b1610 0x7fed700b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.000+0000 7fed77bf7640 1 --2- 172.21.15.163:0/674689065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed7006c860 0x7fed7013f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.000+0000 7fed77bf7640 1 -- 172.21.15.163:0/674689065 >> 172.21.15.163:0/674689065 conn(0x7fed700b3c30 msgr2=0x7fed700aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.001+0000 7fed77bf7640 1 -- 172.21.15.163:0/674689065 shutdown_connections 2024-05-08T06:37:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.001+0000 7fed77bf7640 1 -- 172.21.15.163:0/674689065 wait complete. 2024-05-08T06:37:49.011 INFO:tasks.ceph:Scrubbing osd.4 2024-05-08T06:37:49.011 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-08T06:37:49.114 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.115+0000 7f9359c47640 1 Processor -- start 2024-05-08T06:37:49.115 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.115+0000 7f9359c47640 1 -- start start 2024-05-08T06:37:49.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.116+0000 7f9359c47640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f935415c7d0 0x7f935406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.116+0000 7f9359c47640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f935406f010 0x7f93540b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.116+0000 7f9359c47640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93540b4cb0 con 0x7f93540b4820 2024-05-08T06:37:49.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.116+0000 7f9359c47640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f935406f470 con 0x7f935415c7d0 2024-05-08T06:37:49.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.116+0000 7f9359c47640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f93540b4f80 con 0x7f935406f010 2024-05-08T06:37:49.116 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.116+0000 7f93537fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f935415c7d0 0x7f935406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.116+0000 7f93537fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f935415c7d0 0x7f935406ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34062/0 (socket says 172.21.15.163:34062) 2024-05-08T06:37:49.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.116+0000 7f93537fe640 1 -- 172.21.15.163:0/1881952863 learned_addr learned my addr 172.21.15.163:0/1881952863 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:49.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.117+0000 7f9352ffd640 1 --2- 172.21.15.163:0/1881952863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f935406f010 0x7f93540b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.117+0000 7f93537fe640 1 -- 172.21.15.163:0/1881952863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f935406f010 msgr2=0x7f93540b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:49.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.117+0000 7f93537fe640 1 --2- 172.21.15.163:0/1881952863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f935406f010 0x7f93540b42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.117+0000 7f93537fe640 1 -- 172.21.15.163:0/1881952863 >> v1:172.21.15.192:6789/0 conn(0x7f93540b4820 legacy=0x7f935415f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.117 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.117+0000 7f93537fe640 1 -- 172.21.15.163:0/1881952863 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93540b5260 con 0x7f935415c7d0 2024-05-08T06:37:49.120 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.117+0000 7f93537fe640 1 --2- 172.21.15.163:0/1881952863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f935415c7d0 0x7f935406ead0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9344009e50 tx=0x7f934402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44fd5df5988abc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:49.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.118+0000 7f93527fc640 1 -- 172.21.15.163:0/1881952863 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9344036070 con 0x7f935415c7d0 2024-05-08T06:37:49.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.118+0000 7f93527fc640 1 -- 172.21.15.163:0/1881952863 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9344031600 con 0x7f935415c7d0 2024-05-08T06:37:49.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.118+0000 7f93527fc640 1 -- 172.21.15.163:0/1881952863 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93440319d0 con 0x7f935415c7d0 2024-05-08T06:37:49.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.118+0000 7f9359c47640 1 -- 172.21.15.163:0/1881952863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f935415c7d0 msgr2=0x7f935406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.118+0000 7f9359c47640 1 --2- 172.21.15.163:0/1881952863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f935415c7d0 0x7f935406ead0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9344009e50 tx=0x7f934402a710 comp rx=0 tx=0).stop 2024-05-08T06:37:49.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.118+0000 7f9359c47640 1 -- 172.21.15.163:0/1881952863 shutdown_connections 2024-05-08T06:37:49.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.118+0000 7f9359c47640 1 --2- 172.21.15.163:0/1881952863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f935406f010 0x7f93540b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.121 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.118+0000 7f9359c47640 1 --2- 172.21.15.163:0/1881952863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f935415c7d0 0x7f935406ead0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.118+0000 7f9359c47640 1 -- 172.21.15.163:0/1881952863 >> 172.21.15.163:0/1881952863 conn(0x7f93540b2b40 msgr2=0x7f93540b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:49.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.118+0000 7f9359c47640 1 -- 172.21.15.163:0/1881952863 shutdown_connections 2024-05-08T06:37:49.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.118+0000 7f9359c47640 1 -- 172.21.15.163:0/1881952863 wait complete. 2024-05-08T06:37:49.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.119+0000 7f9359c47640 1 Processor -- start 2024-05-08T06:37:49.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.120+0000 7f9359c47640 1 -- start start 2024-05-08T06:37:49.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.120+0000 7f9359c47640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f935406f010 0x7f935414dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.122 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.120+0000 7f9359c47640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93540b4820 0x7f93540ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.120+0000 7f9359c47640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f935415c7d0 0x7f93540aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.120+0000 7f9359c47640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f935415d630 con 0x7f93540b4820 2024-05-08T06:37:49.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.120+0000 7f9359c47640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f935415d350 con 0x7f935406f010 2024-05-08T06:37:49.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.120+0000 7f9359c47640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f935415cbb0 con 0x7f935415c7d0 2024-05-08T06:37:49.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.120+0000 7f9352ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93540b4820 0x7f93540ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.123 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.120+0000 7f9352ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93540b4820 0x7f93540ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34078/0 (socket says 172.21.15.163:34078) 2024-05-08T06:37:49.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.120+0000 7f9352ffd640 1 -- 172.21.15.163:0/1390227034 learned_addr learned my addr 172.21.15.163:0/1390227034 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:49.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.121+0000 7f9352ffd640 1 -- 172.21.15.163:0/1390227034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f935415c7d0 msgr2=0x7f93540aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.121+0000 7f93537fe640 1 --2- 172.21.15.163:0/1390227034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f935406f010 0x7f935414dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.121+0000 7f9352ffd640 1 --2- 172.21.15.163:0/1390227034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f935415c7d0 0x7f93540aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.121+0000 7f9352ffd640 1 -- 172.21.15.163:0/1390227034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f935406f010 msgr2=0x7f935414dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.124 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.121+0000 7f9352ffd640 1 --2- 172.21.15.163:0/1390227034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f935406f010 0x7f935414dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.121+0000 7f9352ffd640 1 -- 172.21.15.163:0/1390227034 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935409cc30 con 0x7f93540b4820 2024-05-08T06:37:49.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.121+0000 7f9352ffd640 1 --2- 172.21.15.163:0/1390227034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93540b4820 0x7f93540ae7b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f934800d5f0 tx=0x7f934800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:49.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.121+0000 7f933bfff640 1 -- 172.21.15.163:0/1390227034 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93480125f0 con 0x7f93540b4820 2024-05-08T06:37:49.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.121+0000 7f933bfff640 1 -- 172.21.15.163:0/1390227034 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9348019070 con 0x7f93540b4820 2024-05-08T06:37:49.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.121+0000 7f933bfff640 1 -- 172.21.15.163:0/1390227034 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93480129d0 con 0x7f93540b4820 2024-05-08T06:37:49.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.121+0000 7f9359c47640 1 -- 172.21.15.163:0/1390227034 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935409ced0 con 0x7f93540b4820 2024-05-08T06:37:49.125 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.121+0000 7f9359c47640 1 -- 172.21.15.163:0/1390227034 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935409d490 con 0x7f93540b4820 2024-05-08T06:37:49.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.122+0000 7f9359c47640 1 -- 172.21.15.163:0/1390227034 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f935406a550 con 0x7f93540b4820 2024-05-08T06:37:49.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.123+0000 7f933bfff640 1 -- 172.21.15.163:0/1390227034 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9348012b70 con 0x7f93540b4820 2024-05-08T06:37:49.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.123+0000 7f933bfff640 1 --2- 172.21.15.163:0/1390227034 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f93400762d0 0x7f9340078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.123+0000 7f93537fe640 1 --2- 172.21.15.163:0/1390227034 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f93400762d0 0x7f9340078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.123+0000 7f933bfff640 1 -- 172.21.15.163:0/1390227034 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f93480977b0 con 0x7f93540b4820 2024-05-08T06:37:49.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.124+0000 7f933bfff640 1 --2- 172.21.15.163:0/1390227034 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x7f934007c230 0x7f934007e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.124+0000 7f933bfff640 1 -- 172.21.15.163:0/1390227034 --> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9348013e50 con 0x7f934007c230 2024-05-08T06:37:49.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.124+0000 7f933bfff640 1 -- 172.21.15.163:0/1390227034 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f934809c2c0 con 0x7f93540b4820 2024-05-08T06:37:49.128 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.124+0000 7f93537fe640 1 --2- 172.21.15.163:0/1390227034 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f93400762d0 0x7f9340078790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9344033000 tx=0x7f9344009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:49.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.124+0000 7f9353fff640 1 --2- 172.21.15.163:0/1390227034 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x7f934007c230 0x7f934007e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.124+0000 7f9353fff640 1 --2- 172.21.15.163:0/1390227034 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x7f934007c230 0x7f934007e650 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:49.129 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.125+0000 7f933bfff640 1 -- 172.21.15.163:0/1390227034 <== osd.4 v2:172.21.15.192:6824/272496966 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9348013e50 con 0x7f934007c230 2024-05-08T06:37:49.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.138+0000 7f9359c47640 1 -- 172.21.15.163:0/1390227034 --> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f935406f980 con 0x7f934007c230 2024-05-08T06:37:49.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.138+0000 7f933bfff640 1 -- 172.21.15.163:0/1390227034 <== osd.4 v2:172.21.15.192:6824/272496966 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f935406f980 con 0x7f934007c230 2024-05-08T06:37:49.138 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:37:49.138 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T06:37:49.138 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:37:49.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.139+0000 7f9359c47640 1 -- 172.21.15.163:0/1390227034 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x7f934007c230 msgr2=0x7f934007e650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.140 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.139+0000 7f9359c47640 1 --2- 172.21.15.163:0/1390227034 >> [v2:172.21.15.192:6824/272496966,v1:172.21.15.192:6825/272496966] conn(0x7f934007c230 0x7f934007e650 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.139+0000 7f9359c47640 1 -- 172.21.15.163:0/1390227034 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f93400762d0 msgr2=0x7f9340078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.139+0000 7f9359c47640 1 --2- 172.21.15.163:0/1390227034 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f93400762d0 0x7f9340078790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9344033000 tx=0x7f9344009000 comp rx=0 tx=0).stop 2024-05-08T06:37:49.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.139+0000 7f9359c47640 1 -- 172.21.15.163:0/1390227034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93540b4820 msgr2=0x7f93540ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.139+0000 7f9359c47640 1 --2- 172.21.15.163:0/1390227034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93540b4820 0x7f93540ae7b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f934800d5f0 tx=0x7f934800dab0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.141 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.139+0000 7f9353fff640 1 -- 172.21.15.163:0/1390227034 reap_dead start 2024-05-08T06:37:49.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.139+0000 7f9359c47640 1 -- 172.21.15.163:0/1390227034 shutdown_connections 2024-05-08T06:37:49.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.139+0000 7f9359c47640 1 -- 172.21.15.163:0/1390227034 >> 172.21.15.163:0/1390227034 conn(0x7f93540b2b40 msgr2=0x7f9354155570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:49.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.139+0000 7f9359c47640 1 -- 172.21.15.163:0/1390227034 shutdown_connections 2024-05-08T06:37:49.142 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.139+0000 7f9359c47640 1 -- 172.21.15.163:0/1390227034 wait complete. 2024-05-08T06:37:49.148 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-08T06:37:49.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.250+0000 7fac22d90640 1 Processor -- start 2024-05-08T06:37:49.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.251+0000 7fac22d90640 1 -- start start 2024-05-08T06:37:49.252 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.251+0000 7fac22d90640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fac1c15c150 0x7fac1c0b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.252 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.251+0000 7fac22d90640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fac1c0b12f0 0x7fac1c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.252 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.251+0000 7fac22d90640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac1c0afd50 con 0x7fac1c0b5470 2024-05-08T06:37:49.252 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.251+0000 7fac22d90640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fac1c15c530 con 0x7fac1c15c150 2024-05-08T06:37:49.252 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.251+0000 7fac22d90640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fac1c0afff0 con 0x7fac1c0b12f0 2024-05-08T06:37:49.252 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.252+0000 7fac20b2c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fac1c15c150 0x7fac1c0b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.252+0000 7fac2132d640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fac1c0b5470 0x7fac1c073480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:41684/0 (socket says 172.21.15.163:41684) 2024-05-08T06:37:49.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.252+0000 7fac2132d640 1 -- 172.21.15.163:0/1674583653 learned_addr learned my addr 172.21.15.163:0/1674583653 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:49.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.252+0000 7fac1bfff640 1 --2- 172.21.15.163:0/1674583653 >> v2:172.21.15.192:3300/0 conn(0x7fac1c0b12f0 0x7fac1c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.252+0000 7fac20b2c640 1 -- 172.21.15.163:0/1674583653 >> v2:172.21.15.192:3300/0 conn(0x7fac1c0b12f0 msgr2=0x7fac1c0b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:49.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.252+0000 7fac20b2c640 1 --2- 172.21.15.163:0/1674583653 >> v2:172.21.15.192:3300/0 conn(0x7fac1c0b12f0 0x7fac1c0b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.252+0000 7fac20b2c640 1 -- 172.21.15.163:0/1674583653 >> v1:172.21.15.163:6789/0 conn(0x7fac1c0b5470 legacy=0x7fac1c073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.252+0000 7fac20b2c640 1 -- 172.21.15.163:0/1674583653 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c0b02d0 con 0x7fac1c15c150 2024-05-08T06:37:49.253 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.253+0000 7fac20b2c640 1 --2- 172.21.15.163:0/1674583653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c15c150 0x7fac1c0b0db0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0aff20 tx=0x7fac0c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262c49bfb28afa4f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:49.254 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.253+0000 7fac1b7fe640 1 -- 172.21.15.163:0/1674583653 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c007740 con 0x7fac1c15c150 2024-05-08T06:37:49.254 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.253+0000 7fac1b7fe640 1 -- 172.21.15.163:0/1674583653 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac0c0078e0 con 0x7fac1c15c150 2024-05-08T06:37:49.254 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.253+0000 7fac1b7fe640 1 -- 172.21.15.163:0/1674583653 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c007cb0 con 0x7fac1c15c150 2024-05-08T06:37:49.258 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.253+0000 7fac22d90640 1 -- 172.21.15.163:0/1674583653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c15c150 msgr2=0x7fac1c0b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.258 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.253+0000 7fac22d90640 1 --2- 172.21.15.163:0/1674583653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c15c150 0x7fac1c0b0db0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0aff20 tx=0x7fac0c02a660 comp rx=0 tx=0).stop 2024-05-08T06:37:49.258 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.253+0000 7fac22d90640 1 -- 172.21.15.163:0/1674583653 shutdown_connections 2024-05-08T06:37:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.253+0000 7fac22d90640 1 --2- 172.21.15.163:0/1674583653 >> v2:172.21.15.192:3300/0 conn(0x7fac1c0b12f0 0x7fac1c0b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.253+0000 7fac22d90640 1 --2- 172.21.15.163:0/1674583653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c15c150 0x7fac1c0b0db0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.253+0000 7fac22d90640 1 -- 172.21.15.163:0/1674583653 >> 172.21.15.163:0/1674583653 conn(0x7fac1c0b3c30 msgr2=0x7fac1c0b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.253+0000 7fac22d90640 1 -- 172.21.15.163:0/1674583653 shutdown_connections 2024-05-08T06:37:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.253+0000 7fac22d90640 1 -- 172.21.15.163:0/1674583653 wait complete. 2024-05-08T06:37:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.254+0000 7fac22d90640 1 Processor -- start 2024-05-08T06:37:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.255+0000 7fac22d90640 1 -- start start 2024-05-08T06:37:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.255+0000 7fac22d90640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1c0b12f0 0x7fac1c149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.255+0000 7fac22d90640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac1c0b5470 0x7fac1c14fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.260 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.255+0000 7fac22d90640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c15c150 0x7fac1c150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.260 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.255+0000 7fac22d90640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c073b90 con 0x7fac1c15c150 2024-05-08T06:37:49.260 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.255+0000 7fac22d90640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c0b0890 con 0x7fac1c0b12f0 2024-05-08T06:37:49.260 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.255+0000 7fac22d90640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fac1c073d10 con 0x7fac1c0b5470 2024-05-08T06:37:49.260 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac2132d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c15c150 0x7fac1c150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.260 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac2132d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c15c150 0x7fac1c150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34098/0 (socket says 172.21.15.163:34098) 2024-05-08T06:37:49.260 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac2132d640 1 -- 172.21.15.163:0/3961530812 learned_addr learned my addr 172.21.15.163:0/3961530812 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:49.260 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac20b2c640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1c0b12f0 0x7fac1c149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.261 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac1bfff640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac1c0b5470 0x7fac1c14fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.261 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac2132d640 1 -- 172.21.15.163:0/3961530812 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac1c0b5470 msgr2=0x7fac1c14fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.261 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac2132d640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac1c0b5470 0x7fac1c14fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.261 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac2132d640 1 -- 172.21.15.163:0/3961530812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1c0b12f0 msgr2=0x7fac1c149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.261 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac2132d640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1c0b12f0 0x7fac1c149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.261 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac2132d640 1 -- 172.21.15.163:0/3961530812 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c14b890 con 0x7fac1c15c150 2024-05-08T06:37:49.261 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac1bfff640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac1c0b5470 0x7fac1c14fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:49.262 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac2132d640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c15c150 0x7fac1c150390 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fac1400c480 tx=0x7fac1400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:49.262 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac197fa640 1 -- 172.21.15.163:0/3961530812 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14017070 con 0x7fac1c15c150 2024-05-08T06:37:49.262 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac197fa640 1 -- 172.21.15.163:0/3961530812 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac14005ce0 con 0x7fac1c15c150 2024-05-08T06:37:49.262 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac22d90640 1 -- 172.21.15.163:0/3961530812 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c15d610 con 0x7fac1c15c150 2024-05-08T06:37:49.262 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac197fa640 1 -- 172.21.15.163:0/3961530812 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14007270 con 0x7fac1c15c150 2024-05-08T06:37:49.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.256+0000 7fac22d90640 1 -- 172.21.15.163:0/3961530812 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c15dbd0 con 0x7fac1c15c150 2024-05-08T06:37:49.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.257+0000 7fac22d90640 1 -- 172.21.15.163:0/3961530812 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac1c06a4b0 con 0x7fac1c15c150 2024-05-08T06:37:49.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.258+0000 7fac197fa640 1 -- 172.21.15.163:0/3961530812 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fac1401a070 con 0x7fac1c15c150 2024-05-08T06:37:49.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.258+0000 7fac197fa640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fabfc0762d0 0x7fabfc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.259+0000 7fac20b2c640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fabfc0762d0 0x7fabfc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.259+0000 7fac197fa640 1 -- 172.21.15.163:0/3961530812 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fac14096580 con 0x7fac1c15c150 2024-05-08T06:37:49.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.259+0000 7fac20b2c640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fabfc0762d0 0x7fabfc078790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac0c034000 tx=0x7fac0c033040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:49.264 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.263+0000 7fac197fa640 1 -- 172.21.15.163:0/3961530812 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fac1405fbc0 con 0x7fac1c15c150 2024-05-08T06:37:49.449 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.450+0000 7fac22d90640 1 -- 172.21.15.163:0/3961530812 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fac1c14a060 con 0x7fabfc0762d0 2024-05-08T06:37:49.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.450+0000 7fac197fa640 1 -- 172.21.15.163:0/3961530812 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fac0c007840 con 0x7fabfc0762d0 2024-05-08T06:37:49.450 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 4 to deep-scrub 2024-05-08T06:37:49.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.452+0000 7fac22d90640 1 -- 172.21.15.163:0/3961530812 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fabfc0762d0 msgr2=0x7fabfc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.452+0000 7fac22d90640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fabfc0762d0 0x7fabfc078790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac0c034000 tx=0x7fac0c033040 comp rx=0 tx=0).stop 2024-05-08T06:37:49.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.452+0000 7fac22d90640 1 -- 172.21.15.163:0/3961530812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c15c150 msgr2=0x7fac1c150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.452+0000 7fac22d90640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c15c150 0x7fac1c150390 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fac1400c480 tx=0x7fac1400c940 comp rx=0 tx=0).stop 2024-05-08T06:37:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.452+0000 7fac22d90640 1 -- 172.21.15.163:0/3961530812 shutdown_connections 2024-05-08T06:37:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.452+0000 7fac22d90640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fabfc0762d0 0x7fabfc078790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.452+0000 7fac22d90640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c15c150 0x7fac1c150390 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.452+0000 7fac22d90640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac1c0b5470 0x7fac1c14fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.452+0000 7fac22d90640 1 --2- 172.21.15.163:0/3961530812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac1c0b12f0 0x7fac1c149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.452+0000 7fac22d90640 1 -- 172.21.15.163:0/3961530812 >> 172.21.15.163:0/3961530812 conn(0x7fac1c0b3c30 msgr2=0x7fac1c0b5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.452+0000 7fac22d90640 1 -- 172.21.15.163:0/3961530812 shutdown_connections 2024-05-08T06:37:49.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.452+0000 7fac22d90640 1 -- 172.21.15.163:0/3961530812 wait complete. 2024-05-08T06:37:49.462 INFO:tasks.ceph:Scrubbing osd.5 2024-05-08T06:37:49.463 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-08T06:37:49.567 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.568+0000 7f6a7bce8640 1 Processor -- start 2024-05-08T06:37:49.568 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.568+0000 7f6a7bce8640 1 -- start start 2024-05-08T06:37:49.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.569+0000 7f6a7bce8640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6a740d3f20 0x7f6a740d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.569+0000 7f6a7bce8640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a740d3140 con 0x7f6a7415f5b0 2024-05-08T06:37:49.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.569+0000 7f6a7bce8640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a740d3420 con 0x7f6a740b4600 2024-05-08T06:37:49.572 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.569+0000 7f6a7bce8640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6a740b4a90 con 0x7f6a740d3f20 2024-05-08T06:37:49.573 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.569+0000 7f6a79a84640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6a7415f5b0 0x7f6a740d3810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:35296/0 (socket says 172.21.15.163:35296) 2024-05-08T06:37:49.573 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.569+0000 7f6a79a84640 1 -- 172.21.15.163:0/2825051703 learned_addr learned my addr 172.21.15.163:0/2825051703 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:49.573 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a7a285640 1 --2- 172.21.15.163:0/2825051703 >> v2:172.21.15.192:3300/0 conn(0x7f6a740d3f20 0x7f6a740d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.573 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 -- 172.21.15.163:0/2825051703 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 707509956 0 0) 0x7f6a740d3140 con 0x7f6a7415f5b0 2024-05-08T06:37:49.573 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 -- 172.21.15.163:0/2825051703 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a60003520 con 0x7f6a7415f5b0 2024-05-08T06:37:49.573 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 -- 172.21.15.163:0/2825051703 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2835932648 0 0) 0x7f6a60003520 con 0x7f6a7415f5b0 2024-05-08T06:37:49.573 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 -- 172.21.15.163:0/2825051703 >> v2:172.21.15.192:3300/0 conn(0x7f6a740d3f20 msgr2=0x7f6a740d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:49.573 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 --2- 172.21.15.163:0/2825051703 >> v2:172.21.15.192:3300/0 conn(0x7f6a740d3f20 0x7f6a740d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.574 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 -- 172.21.15.163:0/2825051703 >> v1:172.21.15.192:6789/0 conn(0x7f6a740b4600 legacy=0x7f6a7406d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.574 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 -- 172.21.15.163:0/2825051703 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a740d0d80 con 0x7f6a7415f5b0 2024-05-08T06:37:49.574 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 -- 172.21.15.163:0/2825051703 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f6a700016f0 con 0x7f6a7415f5b0 2024-05-08T06:37:49.574 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 -- 172.21.15.163:0/2825051703 >> v1:172.21.15.163:6789/0 conn(0x7f6a7415f5b0 legacy=0x7f6a740d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.574 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 --2- 172.21.15.163:0/2825051703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a60003a50 0x7f6a60003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.574 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 --2- 172.21.15.163:0/2825051703 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a60004800 0x7f6a60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.574 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 --2- 172.21.15.163:0/2825051703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a600053c0 0x7f6a6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.574 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 -- 172.21.15.163:0/2825051703 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6a740b4a90 con 0x7f6a60003a50 2024-05-08T06:37:49.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 -- 172.21.15.163:0/2825051703 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6000c230 con 0x7f6a600053c0 2024-05-08T06:37:49.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a78a82640 1 -- 172.21.15.163:0/2825051703 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6a6000c430 con 0x7f6a60004800 2024-05-08T06:37:49.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a79a84640 1 --2- 172.21.15.163:0/2825051703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a60003a50 0x7f6a60003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a7a285640 1 --2- 172.21.15.163:0/2825051703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a600053c0 0x7f6a6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a79283640 1 --2- 172.21.15.163:0/2825051703 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a60004800 0x7f6a60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a79a84640 1 -- 172.21.15.163:0/2825051703 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a60004800 msgr2=0x7f6a60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:49.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a79a84640 1 --2- 172.21.15.163:0/2825051703 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a60004800 0x7f6a60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.576 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a79a84640 1 -- 172.21.15.163:0/2825051703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a600053c0 msgr2=0x7f6a6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:49.576 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a79a84640 1 --2- 172.21.15.163:0/2825051703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a600053c0 0x7f6a6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.576 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a79a84640 1 -- 172.21.15.163:0/2825051703 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a60003520 con 0x7f6a60003a50 2024-05-08T06:37:49.576 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.570+0000 7f6a7a285640 1 -- 172.21.15.163:0/2825051703 reap_dead start 2024-05-08T06:37:49.576 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.571+0000 7f6a79a84640 1 --2- 172.21.15.163:0/2825051703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a60003a50 0x7f6a60003e70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c240 tx=0x7f6a60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3741297853e1c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:49.576 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.571+0000 7f6a78a82640 1 -- 172.21.15.163:0/2825051703 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70007ce0 con 0x7f6a60003a50 2024-05-08T06:37:49.576 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.571+0000 7f6a78a82640 1 -- 172.21.15.163:0/2825051703 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a70005ad0 con 0x7f6a60003a50 2024-05-08T06:37:49.576 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.571+0000 7f6a78a82640 1 -- 172.21.15.163:0/2825051703 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70007070 con 0x7f6a60003a50 2024-05-08T06:37:49.577 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.572+0000 7f6a7bce8640 1 -- 172.21.15.163:0/2825051703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a60003a50 msgr2=0x7f6a60003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.577 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.572+0000 7f6a7bce8640 1 --2- 172.21.15.163:0/2825051703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a60003a50 0x7f6a60003e70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c240 tx=0x7f6a60005830 comp rx=0 tx=0).stop 2024-05-08T06:37:49.577 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.572+0000 7f6a7bce8640 1 -- 172.21.15.163:0/2825051703 shutdown_connections 2024-05-08T06:37:49.577 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.572+0000 7f6a7bce8640 1 --2- 172.21.15.163:0/2825051703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a60003a50 0x7f6a60003e70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.577 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.572+0000 7f6a7bce8640 1 -- 172.21.15.163:0/2825051703 >> 172.21.15.163:0/2825051703 conn(0x7f6a740d5230 msgr2=0x7f6a740d5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:49.577 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.572+0000 7f6a7bce8640 1 -- 172.21.15.163:0/2825051703 shutdown_connections 2024-05-08T06:37:49.577 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.572+0000 7f6a7bce8640 1 -- 172.21.15.163:0/2825051703 wait complete. 2024-05-08T06:37:49.577 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.573+0000 7f6a7bce8640 1 Processor -- start 2024-05-08T06:37:49.578 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.573+0000 7f6a7bce8640 1 -- start start 2024-05-08T06:37:49.581 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a7bce8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a60003a50 0x7f6a740ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.581 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a7bce8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a741f84a0 0x7f6a741f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.581 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a7bce8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a740ba020 0x7f6a740ba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.581 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a7bce8640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6a740d5fd0 con 0x7f6a60003a50 2024-05-08T06:37:49.581 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a7bce8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6a740d0b30 con 0x7f6a741f84a0 2024-05-08T06:37:49.581 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a7bce8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6a740d6150 con 0x7f6a740ba020 2024-05-08T06:37:49.582 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a79a84640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a60003a50 0x7f6a740ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.582 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a79a84640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a60003a50 0x7f6a740ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35074/0 (socket says 172.21.15.163:35074) 2024-05-08T06:37:49.582 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a79a84640 1 -- 172.21.15.163:0/4142200652 learned_addr learned my addr 172.21.15.163:0/4142200652 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:49.582 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a79283640 1 --2- 172.21.15.163:0/4142200652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a741f84a0 0x7f6a741f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.582 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a7a285640 1 --2- 172.21.15.163:0/4142200652 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a740ba020 0x7f6a740ba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.582 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a79a84640 1 -- 172.21.15.163:0/4142200652 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a740ba020 msgr2=0x7f6a740ba4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a79a84640 1 --2- 172.21.15.163:0/4142200652 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a740ba020 0x7f6a740ba4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a79a84640 1 -- 172.21.15.163:0/4142200652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a741f84a0 msgr2=0x7f6a741f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a79a84640 1 --2- 172.21.15.163:0/4142200652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a741f84a0 0x7f6a741f8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a79a84640 1 -- 172.21.15.163:0/4142200652 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a741f92f0 con 0x7f6a60003a50 2024-05-08T06:37:49.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a79a84640 1 --2- 172.21.15.163:0/4142200652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a60003a50 0x7f6a740ce7c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6a7002ad20 tx=0x7f6a740d35f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:49.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a6a7fc640 1 -- 172.21.15.163:0/4142200652 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a700016f0 con 0x7f6a60003a50 2024-05-08T06:37:49.584 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a6a7fc640 1 -- 172.21.15.163:0/4142200652 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a70042070 con 0x7f6a60003a50 2024-05-08T06:37:49.584 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a7bce8640 1 -- 172.21.15.163:0/4142200652 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a741f9010 con 0x7f6a60003a50 2024-05-08T06:37:49.584 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.574+0000 7f6a6a7fc640 1 -- 172.21.15.163:0/4142200652 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a700072b0 con 0x7f6a60003a50 2024-05-08T06:37:49.584 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.575+0000 7f6a7bce8640 1 -- 172.21.15.163:0/4142200652 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a741f52a0 con 0x7f6a60003a50 2024-05-08T06:37:49.584 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.575+0000 7f6a7bce8640 1 -- 172.21.15.163:0/4142200652 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6a7406a550 con 0x7f6a60003a50 2024-05-08T06:37:49.584 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.576+0000 7f6a6a7fc640 1 -- 172.21.15.163:0/4142200652 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6a70007450 con 0x7f6a60003a50 2024-05-08T06:37:49.585 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.577+0000 7f6a6a7fc640 1 --2- 172.21.15.163:0/4142200652 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6a580762d0 0x7f6a58078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.585 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.577+0000 7f6a6a7fc640 1 -- 172.21.15.163:0/4142200652 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f6a700b7d60 con 0x7f6a60003a50 2024-05-08T06:37:49.585 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.577+0000 7f6a79283640 1 --2- 172.21.15.163:0/4142200652 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6a580762d0 0x7f6a58078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.585 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.577+0000 7f6a6a7fc640 1 --2- 172.21.15.163:0/4142200652 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x7f6a5807c230 0x7f6a5807e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.585 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.577+0000 7f6a6a7fc640 1 -- 172.21.15.163:0/4142200652 --> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a7002ee50 con 0x7f6a5807c230 2024-05-08T06:37:49.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.577+0000 7f6a6a7fc640 1 -- 172.21.15.163:0/4142200652 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6a700b81f0 con 0x7f6a60003a50 2024-05-08T06:37:49.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.577+0000 7f6a7a285640 1 --2- 172.21.15.163:0/4142200652 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x7f6a5807c230 0x7f6a5807e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.577+0000 7f6a79283640 1 --2- 172.21.15.163:0/4142200652 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6a580762d0 0x7f6a58078790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6a740cf6c0 tx=0x7f6a64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:49.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.577+0000 7f6a7a285640 1 --2- 172.21.15.163:0/4142200652 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x7f6a5807c230 0x7f6a5807e650 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:49.586 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.578+0000 7f6a6a7fc640 1 -- 172.21.15.163:0/4142200652 <== osd.5 v2:172.21.15.192:6800/2911825781 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6a7002ee50 con 0x7f6a5807c230 2024-05-08T06:37:49.590 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.591+0000 7f6a7bce8640 1 -- 172.21.15.163:0/4142200652 --> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6a7415dc50 con 0x7f6a5807c230 2024-05-08T06:37:49.591 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.592+0000 7f6a6a7fc640 1 -- 172.21.15.163:0/4142200652 <== osd.5 v2:172.21.15.192:6800/2911825781 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6a7415dc50 con 0x7f6a5807c230 2024-05-08T06:37:49.591 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:37:49.592 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T06:37:49.592 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:37:49.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.592+0000 7f6a7bce8640 1 -- 172.21.15.163:0/4142200652 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x7f6a5807c230 msgr2=0x7f6a5807e650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.592+0000 7f6a7bce8640 1 --2- 172.21.15.163:0/4142200652 >> [v2:172.21.15.192:6800/2911825781,v1:172.21.15.192:6801/2911825781] conn(0x7f6a5807c230 0x7f6a5807e650 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.593 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.592+0000 7f6a7bce8640 1 -- 172.21.15.163:0/4142200652 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6a580762d0 msgr2=0x7f6a58078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.592+0000 7f6a7bce8640 1 --2- 172.21.15.163:0/4142200652 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f6a580762d0 0x7f6a58078790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6a740cf6c0 tx=0x7f6a64009040 comp rx=0 tx=0).stop 2024-05-08T06:37:49.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.592+0000 7f6a7bce8640 1 -- 172.21.15.163:0/4142200652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a60003a50 msgr2=0x7f6a740ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.592+0000 7f6a7bce8640 1 --2- 172.21.15.163:0/4142200652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a60003a50 0x7f6a740ce7c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6a7002ad20 tx=0x7f6a740d35f0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.592+0000 7f6a7a285640 1 -- 172.21.15.163:0/4142200652 reap_dead start 2024-05-08T06:37:49.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.592+0000 7f6a7bce8640 1 -- 172.21.15.163:0/4142200652 shutdown_connections 2024-05-08T06:37:49.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.592+0000 7f6a7bce8640 1 -- 172.21.15.163:0/4142200652 >> 172.21.15.163:0/4142200652 conn(0x7f6a740d5230 msgr2=0x7f6a7415c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:49.594 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.593+0000 7f6a7bce8640 1 -- 172.21.15.163:0/4142200652 shutdown_connections 2024-05-08T06:37:49.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.593+0000 7f6a7bce8640 1 -- 172.21.15.163:0/4142200652 wait complete. 2024-05-08T06:37:49.602 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-08T06:37:49.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.704+0000 7f2453566640 1 Processor -- start 2024-05-08T06:37:49.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.704+0000 7f2453566640 1 -- start start 2024-05-08T06:37:49.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.705+0000 7f2453566640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f244c0b8240 0x7f244c0d3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.705+0000 7f2453566640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f244c0b8a60 con 0x7f244c0d2250 2024-05-08T06:37:49.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.705+0000 7f2453566640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f244c0b8d40 con 0x7f244c15f350 2024-05-08T06:37:49.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.705+0000 7f2453566640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f244c0b9670 con 0x7f244c0b8240 2024-05-08T06:37:49.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.705+0000 7f2450b01640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f244c0b8240 0x7f244c0d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.705+0000 7f2451b03640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f244c0d2250 0x7f244c0b9b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:35308/0 (socket says 172.21.15.163:35308) 2024-05-08T06:37:49.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.706+0000 7f2451b03640 1 -- 172.21.15.163:0/1683750465 learned_addr learned my addr 172.21.15.163:0/1683750465 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:49.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.706+0000 7f2450b01640 1 -- 172.21.15.163:0/1683750465 >> v1:172.21.15.192:6789/0 conn(0x7f244c15f350 legacy=0x7f244c0bace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.706+0000 7f2450b01640 1 -- 172.21.15.163:0/1683750465 >> v1:172.21.15.163:6789/0 conn(0x7f244c0d2250 legacy=0x7f244c0b9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.706+0000 7f2450b01640 1 -- 172.21.15.163:0/1683750465 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244c0b9020 con 0x7f244c0b8240 2024-05-08T06:37:49.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.706+0000 7f2450b01640 1 --2- 172.21.15.163:0/1683750465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f244c0b8240 0x7f244c0d3580 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f243c0097a0 tx=0x7f243c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751df5202f1ccdd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:49.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.707+0000 7f2443fff640 1 -- 172.21.15.163:0/1683750465 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c037070 con 0x7f244c0b8240 2024-05-08T06:37:49.708 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.707+0000 7f2443fff640 1 -- 172.21.15.163:0/1683750465 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f243c032640 con 0x7f244c0b8240 2024-05-08T06:37:49.708 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.707+0000 7f2443fff640 1 -- 172.21.15.163:0/1683750465 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c032a10 con 0x7f244c0b8240 2024-05-08T06:37:49.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.707+0000 7f2453566640 1 -- 172.21.15.163:0/1683750465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f244c0b8240 msgr2=0x7f244c0d3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.707+0000 7f2453566640 1 --2- 172.21.15.163:0/1683750465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f244c0b8240 0x7f244c0d3580 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f243c0097a0 tx=0x7f243c02a2c0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.707+0000 7f2453566640 1 -- 172.21.15.163:0/1683750465 shutdown_connections 2024-05-08T06:37:49.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.707+0000 7f2453566640 1 --2- 172.21.15.163:0/1683750465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f244c0b8240 0x7f244c0d3580 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.707+0000 7f2453566640 1 -- 172.21.15.163:0/1683750465 >> 172.21.15.163:0/1683750465 conn(0x7f244c073e70 msgr2=0x7f244c074280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:49.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.707+0000 7f2453566640 1 -- 172.21.15.163:0/1683750465 shutdown_connections 2024-05-08T06:37:49.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.707+0000 7f2453566640 1 -- 172.21.15.163:0/1683750465 wait complete. 2024-05-08T06:37:49.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.708+0000 7f2453566640 1 Processor -- start 2024-05-08T06:37:49.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.709+0000 7f2453566640 1 -- start start 2024-05-08T06:37:49.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.709+0000 7f2453566640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f244c0b8240 0x7f244c161d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.709+0000 7f2453566640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f244c0d2250 0x7f244c15f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.709+0000 7f2453566640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f244c15f350 0x7f244c15fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.709+0000 7f2453566640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f244c15f730 con 0x7f244c0b8240 2024-05-08T06:37:49.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.709+0000 7f2453566640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f244c0d3ac0 con 0x7f244c15f350 2024-05-08T06:37:49.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.709+0000 7f2453566640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f244c0b86a0 con 0x7f244c0d2250 2024-05-08T06:37:49.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.709+0000 7f2451302640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f244c0b8240 0x7f244c161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.709+0000 7f2451302640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f244c0b8240 0x7f244c161d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35092/0 (socket says 172.21.15.163:35092) 2024-05-08T06:37:49.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.709+0000 7f2451302640 1 -- 172.21.15.163:0/209541236 learned_addr learned my addr 172.21.15.163:0/209541236 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:49.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.709+0000 7f2451b03640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f244c15f350 0x7f244c15fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.710+0000 7f2450b01640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f244c0d2250 0x7f244c15f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.710+0000 7f2451302640 1 -- 172.21.15.163:0/209541236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f244c0d2250 msgr2=0x7f244c15f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.715 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.710+0000 7f2451302640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f244c0d2250 0x7f244c15f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.710+0000 7f2451302640 1 -- 172.21.15.163:0/209541236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f244c15f350 msgr2=0x7f244c15fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.710+0000 7f2451302640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f244c15f350 0x7f244c15fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.710+0000 7f2451302640 1 -- 172.21.15.163:0/209541236 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244c1605e0 con 0x7f244c0b8240 2024-05-08T06:37:49.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.710+0000 7f2450b01640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f244c0d2250 0x7f244c15f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:37:49.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.710+0000 7f2451302640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f244c0b8240 0x7f244c161d00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f244800c450 tx=0x7f244800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:49.716 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.710+0000 7f2441ffb640 1 -- 172.21.15.163:0/209541236 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24480052d0 con 0x7f244c0b8240 2024-05-08T06:37:49.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.710+0000 7f2441ffb640 1 -- 172.21.15.163:0/209541236 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2448005470 con 0x7f244c0b8240 2024-05-08T06:37:49.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.710+0000 7f2453566640 1 -- 172.21.15.163:0/209541236 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244c1608c0 con 0x7f244c0b8240 2024-05-08T06:37:49.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.710+0000 7f2441ffb640 1 -- 172.21.15.163:0/209541236 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448012870 con 0x7f244c0b8240 2024-05-08T06:37:49.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.710+0000 7f2453566640 1 -- 172.21.15.163:0/209541236 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244c160e80 con 0x7f244c0b8240 2024-05-08T06:37:49.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.711+0000 7f2453566640 1 -- 172.21.15.163:0/209541236 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f244c06a4b0 con 0x7f244c0b8240 2024-05-08T06:37:49.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.712+0000 7f2441ffb640 1 -- 172.21.15.163:0/209541236 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2448012a10 con 0x7f244c0b8240 2024-05-08T06:37:49.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.712+0000 7f2441ffb640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f24300762d0 0x7f2430078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:49.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.712+0000 7f2450b01640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f24300762d0 0x7f2430078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:49.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.712+0000 7f2441ffb640 1 -- 172.21.15.163:0/209541236 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f2448096ee0 con 0x7f244c0b8240 2024-05-08T06:37:49.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.713+0000 7f2450b01640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f24300762d0 0x7f2430078790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f243c034000 tx=0x7f243c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:49.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.716+0000 7f2441ffb640 1 -- 172.21.15.163:0/209541236 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2448060b70 con 0x7f244c0b8240 2024-05-08T06:37:49.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.899+0000 7f2453566640 1 -- 172.21.15.163:0/209541236 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f244c0b4610 con 0x7f24300762d0 2024-05-08T06:37:49.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.899+0000 7f2441ffb640 1 -- 172.21.15.163:0/209541236 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f244c0b4610 con 0x7f24300762d0 2024-05-08T06:37:49.899 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 5 to deep-scrub 2024-05-08T06:37:49.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.901+0000 7f2453566640 1 -- 172.21.15.163:0/209541236 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f24300762d0 msgr2=0x7f2430078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.901+0000 7f2453566640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f24300762d0 0x7f2430078790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f243c034000 tx=0x7f243c02e000 comp rx=0 tx=0).stop 2024-05-08T06:37:49.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.901+0000 7f2453566640 1 -- 172.21.15.163:0/209541236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f244c0b8240 msgr2=0x7f244c161d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:49.901 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.901+0000 7f2453566640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f244c0b8240 0x7f244c161d00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f244800c450 tx=0x7f244800c910 comp rx=0 tx=0).stop 2024-05-08T06:37:49.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.901+0000 7f2453566640 1 -- 172.21.15.163:0/209541236 shutdown_connections 2024-05-08T06:37:49.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.901+0000 7f2453566640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f24300762d0 0x7f2430078790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.901+0000 7f2453566640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f244c15f350 0x7f244c15fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.901+0000 7f2453566640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f244c0d2250 0x7f244c15f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.901+0000 7f2453566640 1 --2- 172.21.15.163:0/209541236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f244c0b8240 0x7f244c161d00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:49.902 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.901+0000 7f2453566640 1 -- 172.21.15.163:0/209541236 >> 172.21.15.163:0/209541236 conn(0x7f244c073e70 msgr2=0x7f244c0d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:49.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.902+0000 7f2453566640 1 -- 172.21.15.163:0/209541236 shutdown_connections 2024-05-08T06:37:49.903 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:49.902+0000 7f2453566640 1 -- 172.21.15.163:0/209541236 wait complete. 2024-05-08T06:37:49.912 INFO:tasks.ceph:Scrubbing osd.6 2024-05-08T06:37:49.912 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-08T06:37:50.016 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.016+0000 7fa2956c8640 1 Processor -- start 2024-05-08T06:37:50.016 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.016+0000 7fa2956c8640 1 -- start start 2024-05-08T06:37:50.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.017+0000 7fa2956c8640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa2900b4e60 0x7fa2900afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.017+0000 7fa2956c8640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa2900b0000 0x7fa29006e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.017+0000 7fa2956c8640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2900ae160 con 0x7fa2901578e0 2024-05-08T06:37:50.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.017+0000 7fa2956c8640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa2900b4410 con 0x7fa2900b4e60 2024-05-08T06:37:50.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.017+0000 7fa2956c8640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa2900ae420 con 0x7fa2900b0000 2024-05-08T06:37:50.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.017+0000 7fa287fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa2900b4e60 0x7fa2900afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.019 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.017+0000 7fa28effd640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa2901578e0 0x7fa29015e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:35318/0 (socket says 172.21.15.163:35318) 2024-05-08T06:37:50.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.017+0000 7fa28effd640 1 -- 172.21.15.163:0/3190180284 learned_addr learned my addr 172.21.15.163:0/3190180284 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:50.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.018+0000 7fa28f7fe640 1 --2- 172.21.15.163:0/3190180284 >> v2:172.21.15.192:3300/0 conn(0x7fa2900b0000 0x7fa29006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.018+0000 7fa287fff640 1 -- 172.21.15.163:0/3190180284 >> v2:172.21.15.192:3300/0 conn(0x7fa2900b0000 msgr2=0x7fa29006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:50.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.018+0000 7fa287fff640 1 --2- 172.21.15.163:0/3190180284 >> v2:172.21.15.192:3300/0 conn(0x7fa2900b0000 0x7fa29006e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.020 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.018+0000 7fa28e7fc640 1 -- 172.21.15.163:0/3190180284 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4236755097 0 0) 0x7fa2900ae160 con 0x7fa2901578e0 2024-05-08T06:37:50.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.018+0000 7fa287fff640 1 -- 172.21.15.163:0/3190180284 >> v1:172.21.15.163:6789/0 conn(0x7fa2901578e0 legacy=0x7fa29015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.018+0000 7fa287fff640 1 -- 172.21.15.163:0/3190180284 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2900b4660 con 0x7fa2900b4e60 2024-05-08T06:37:50.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.018+0000 7fa287fff640 1 --2- 172.21.15.163:0/3190180284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2900b4e60 0x7fa2900afac0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa2900d0d30 tx=0x7fa27802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c1e5150c49c651 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.019+0000 7fa28e7fc640 1 -- 172.21.15.163:0/3190180284 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278037070 con 0x7fa2900b4e60 2024-05-08T06:37:50.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.019+0000 7fa28e7fc640 1 -- 172.21.15.163:0/3190180284 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa27802add0 con 0x7fa2900b4e60 2024-05-08T06:37:50.021 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.019+0000 7fa28e7fc640 1 -- 172.21.15.163:0/3190180284 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278032660 con 0x7fa2900b4e60 2024-05-08T06:37:50.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.019+0000 7fa2956c8640 1 -- 172.21.15.163:0/3190180284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2900b4e60 msgr2=0x7fa2900afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.019+0000 7fa2956c8640 1 --2- 172.21.15.163:0/3190180284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2900b4e60 0x7fa2900afac0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa2900d0d30 tx=0x7fa27802a080 comp rx=0 tx=0).stop 2024-05-08T06:37:50.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.019+0000 7fa2956c8640 1 -- 172.21.15.163:0/3190180284 shutdown_connections 2024-05-08T06:37:50.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.019+0000 7fa2956c8640 1 --2- 172.21.15.163:0/3190180284 >> v2:172.21.15.192:3300/0 conn(0x7fa2900b0000 0x7fa29006e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.022 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.019+0000 7fa2956c8640 1 --2- 172.21.15.163:0/3190180284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2900b4e60 0x7fa2900afac0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.019+0000 7fa2956c8640 1 -- 172.21.15.163:0/3190180284 >> 172.21.15.163:0/3190180284 conn(0x7fa2900b2b40 msgr2=0x7fa2900b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:50.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.019+0000 7fa2956c8640 1 -- 172.21.15.163:0/3190180284 shutdown_connections 2024-05-08T06:37:50.023 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.019+0000 7fa2956c8640 1 -- 172.21.15.163:0/3190180284 wait complete. 2024-05-08T06:37:50.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.020+0000 7fa2956c8640 1 Processor -- start 2024-05-08T06:37:50.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.020+0000 7fa2956c8640 1 -- start start 2024-05-08T06:37:50.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa2956c8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2900b0000 0x7fa290143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa2956c8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2900b4e60 0x7fa2901415a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa2956c8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2901578e0 0x7fa290141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa2956c8640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa2900b0410 con 0x7fa2900b0000 2024-05-08T06:37:50.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa2956c8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa2900ade20 con 0x7fa2901578e0 2024-05-08T06:37:50.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa2956c8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa2900b6260 con 0x7fa2900b4e60 2024-05-08T06:37:50.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa28effd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2900b0000 0x7fa290143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa28effd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2900b0000 0x7fa290143370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35114/0 (socket says 172.21.15.163:35114) 2024-05-08T06:37:50.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa28effd640 1 -- 172.21.15.163:0/2480660846 learned_addr learned my addr 172.21.15.163:0/2480660846 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:50.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa287fff640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2900b4e60 0x7fa2901415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa28f7fe640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2901578e0 0x7fa290141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa28effd640 1 -- 172.21.15.163:0/2480660846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2900b4e60 msgr2=0x7fa2901415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa28effd640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2900b4e60 0x7fa2901415a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa28effd640 1 -- 172.21.15.163:0/2480660846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2901578e0 msgr2=0x7fa290141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa28effd640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2901578e0 0x7fa290141ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa28effd640 1 -- 172.21.15.163:0/2480660846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa290142020 con 0x7fa2900b0000 2024-05-08T06:37:50.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.021+0000 7fa287fff640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2900b4e60 0x7fa2901415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:50.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.022+0000 7fa28effd640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2900b0000 0x7fa290143370 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa28000c4d0 tx=0x7fa28000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.022+0000 7fa2877fe640 1 -- 172.21.15.163:0/2480660846 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa280017070 con 0x7fa2900b0000 2024-05-08T06:37:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.022+0000 7fa2877fe640 1 -- 172.21.15.163:0/2480660846 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa280005d00 con 0x7fa2900b0000 2024-05-08T06:37:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.022+0000 7fa2956c8640 1 -- 172.21.15.163:0/2480660846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2901422d0 con 0x7fa2900b0000 2024-05-08T06:37:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.022+0000 7fa2877fe640 1 -- 172.21.15.163:0/2480660846 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2800072b0 con 0x7fa2900b0000 2024-05-08T06:37:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.022+0000 7fa2956c8640 1 -- 172.21.15.163:0/2480660846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa290142890 con 0x7fa2900b0000 2024-05-08T06:37:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.023+0000 7fa2956c8640 1 -- 172.21.15.163:0/2480660846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa29006a550 con 0x7fa2900b0000 2024-05-08T06:37:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.023+0000 7fa2877fe640 1 -- 172.21.15.163:0/2480660846 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa28001a070 con 0x7fa2900b0000 2024-05-08T06:37:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.024+0000 7fa2877fe640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa270076300 0x7fa2700787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.024+0000 7fa2877fe640 1 -- 172.21.15.163:0/2480660846 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fa280096660 con 0x7fa2900b0000 2024-05-08T06:37:50.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.024+0000 7fa287fff640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa270076300 0x7fa2700787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.024+0000 7fa2877fe640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x7fa27007c260 0x7fa27007e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.024+0000 7fa2877fe640 1 -- 172.21.15.163:0/2480660846 --> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa28001ee50 con 0x7fa27007c260 2024-05-08T06:37:50.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.024+0000 7fa2877fe640 1 -- 172.21.15.163:0/2480660846 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa28009d050 con 0x7fa2900b0000 2024-05-08T06:37:50.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.024+0000 7fa28f7fe640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x7fa27007c260 0x7fa27007e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.024+0000 7fa287fff640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa270076300 0x7fa2700787c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa278034000 tx=0x7fa27802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.025+0000 7fa28f7fe640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x7fa27007c260 0x7fa27007e680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.026+0000 7fa2877fe640 1 -- 172.21.15.163:0/2480660846 <== osd.6 v2:172.21.15.192:6816/1074108675 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa28001ee50 con 0x7fa27007c260 2024-05-08T06:37:50.037 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.038+0000 7fa2956c8640 1 -- 172.21.15.163:0/2480660846 --> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa29006fec0 con 0x7fa27007c260 2024-05-08T06:37:50.038 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.039+0000 7fa2877fe640 1 -- 172.21.15.163:0/2480660846 <== osd.6 v2:172.21.15.192:6816/1074108675 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa29006fec0 con 0x7fa27007c260 2024-05-08T06:37:50.038 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:37:50.039 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T06:37:50.039 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:37:50.040 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.040+0000 7fa2956c8640 1 -- 172.21.15.163:0/2480660846 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x7fa27007c260 msgr2=0x7fa27007e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.040 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.040+0000 7fa2956c8640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.192:6816/1074108675,v1:172.21.15.192:6817/1074108675] conn(0x7fa27007c260 0x7fa27007e680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.040 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.040+0000 7fa2956c8640 1 -- 172.21.15.163:0/2480660846 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa270076300 msgr2=0x7fa2700787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.040 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.040+0000 7fa2956c8640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa270076300 0x7fa2700787c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa278034000 tx=0x7fa27802e000 comp rx=0 tx=0).stop 2024-05-08T06:37:50.040 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.040+0000 7fa2956c8640 1 -- 172.21.15.163:0/2480660846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2900b0000 msgr2=0x7fa290143370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.041 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.040+0000 7fa2956c8640 1 --2- 172.21.15.163:0/2480660846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2900b0000 0x7fa290143370 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa28000c4d0 tx=0x7fa28000c990 comp rx=0 tx=0).stop 2024-05-08T06:37:50.041 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.040+0000 7fa28f7fe640 1 -- 172.21.15.163:0/2480660846 reap_dead start 2024-05-08T06:37:50.041 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.040+0000 7fa2956c8640 1 -- 172.21.15.163:0/2480660846 shutdown_connections 2024-05-08T06:37:50.041 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.040+0000 7fa2956c8640 1 -- 172.21.15.163:0/2480660846 >> 172.21.15.163:0/2480660846 conn(0x7fa2900b2b40 msgr2=0x7fa2900b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:50.041 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.040+0000 7fa2956c8640 1 -- 172.21.15.163:0/2480660846 shutdown_connections 2024-05-08T06:37:50.041 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.040+0000 7fa2956c8640 1 -- 172.21.15.163:0/2480660846 wait complete. 2024-05-08T06:37:50.049 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-08T06:37:50.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.153+0000 7f76bd578640 1 Processor -- start 2024-05-08T06:37:50.153 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.153+0000 7f76bd578640 1 -- start start 2024-05-08T06:37:50.160 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.154+0000 7f76bd578640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76b806ca30 0x7f76b80b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.160 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.154+0000 7f76bd578640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b815dbc0 con 0x7f76b80b1250 2024-05-08T06:37:50.160 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.154+0000 7f76bd578640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b815dea0 con 0x7f76b8070eb0 2024-05-08T06:37:50.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.154+0000 7f76bd578640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f76b809ffe0 con 0x7f76b806ca30 2024-05-08T06:37:50.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.154+0000 7f76b77fe640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f76b80b1250 0x7f76b8073480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:35332/0 (socket says 172.21.15.163:35332) 2024-05-08T06:37:50.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.154+0000 7f76b77fe640 1 -- 172.21.15.163:0/1828765617 learned_addr learned my addr 172.21.15.163:0/1828765617 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:50.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76ae7fc640 1 --2- 172.21.15.163:0/1828765617 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76b806ca30 0x7f76b80b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 -- 172.21.15.163:0/1828765617 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2841553995 0 0) 0x7f76b815dbc0 con 0x7f76b80b1250 2024-05-08T06:37:50.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 -- 172.21.15.163:0/1828765617 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76a4003520 con 0x7f76b80b1250 2024-05-08T06:37:50.161 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 -- 172.21.15.163:0/1828765617 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1351876568 0 0) 0x7f76a4003520 con 0x7f76b80b1250 2024-05-08T06:37:50.162 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 -- 172.21.15.163:0/1828765617 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76b806ca30 msgr2=0x7f76b80b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:50.162 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 --2- 172.21.15.163:0/1828765617 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76b806ca30 0x7f76b80b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.162 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 -- 172.21.15.163:0/1828765617 >> v1:172.21.15.192:6789/0 conn(0x7f76b8070eb0 legacy=0x7f76b806c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.162 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 -- 172.21.15.163:0/1828765617 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b815e180 con 0x7f76b80b1250 2024-05-08T06:37:50.162 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 -- 172.21.15.163:0/1828765617 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7f76b0001a70 con 0x7f76b80b1250 2024-05-08T06:37:50.163 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 -- 172.21.15.163:0/1828765617 >> v1:172.21.15.163:6789/0 conn(0x7f76b80b1250 legacy=0x7f76b8073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.163 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 --2- 172.21.15.163:0/1828765617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a4003a30 0x7f76a4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.163 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 --2- 172.21.15.163:0/1828765617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a4004800 0x7f76a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.163 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 --2- 172.21.15.163:0/1828765617 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76b806ca30 0x7f76a4005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.163 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 -- 172.21.15.163:0/1828765617 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f76a400bbb0 con 0x7f76a4004800 2024-05-08T06:37:50.163 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 -- 172.21.15.163:0/1828765617 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f76a400bd60 con 0x7f76a4003a30 2024-05-08T06:37:50.163 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b67fc640 1 -- 172.21.15.163:0/1828765617 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f76a400bf60 con 0x7f76b806ca30 2024-05-08T06:37:50.164 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76ae7fc640 1 --2- 172.21.15.163:0/1828765617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a4004800 0x7f76a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.164 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b6ffd640 1 --2- 172.21.15.163:0/1828765617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a4003a30 0x7f76a4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.164 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76b77fe640 1 --2- 172.21.15.163:0/1828765617 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76b806ca30 0x7f76a4005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.164 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76ae7fc640 1 -- 172.21.15.163:0/1828765617 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76b806ca30 msgr2=0x7f76a4005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:50.164 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76ae7fc640 1 --2- 172.21.15.163:0/1828765617 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76b806ca30 0x7f76a4005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.164 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76ae7fc640 1 -- 172.21.15.163:0/1828765617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a4003a30 msgr2=0x7f76a4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:50.164 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76ae7fc640 1 --2- 172.21.15.163:0/1828765617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a4003a30 0x7f76a4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.164 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.155+0000 7f76ae7fc640 1 -- 172.21.15.163:0/1828765617 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a4003520 con 0x7f76a4004800 2024-05-08T06:37:50.164 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.156+0000 7f76ae7fc640 1 --2- 172.21.15.163:0/1828765617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a4004800 0x7f76a4004c80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009ac0 tx=0x7f76a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ecc5d6ffae3b39e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.156+0000 7f76b67fc640 1 -- 172.21.15.163:0/1828765617 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0036070 con 0x7f76a4004800 2024-05-08T06:37:50.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.156+0000 7f76b67fc640 1 -- 172.21.15.163:0/1828765617 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a0031400 con 0x7f76a4004800 2024-05-08T06:37:50.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.156+0000 7f76b67fc640 1 -- 172.21.15.163:0/1828765617 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0009690 con 0x7f76a4004800 2024-05-08T06:37:50.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.157+0000 7f76bd578640 1 -- 172.21.15.163:0/1828765617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a4004800 msgr2=0x7f76a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.157+0000 7f76bd578640 1 --2- 172.21.15.163:0/1828765617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a4004800 0x7f76a4004c80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009ac0 tx=0x7f76a4005830 comp rx=0 tx=0).stop 2024-05-08T06:37:50.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.157+0000 7f76b77fe640 1 -- 172.21.15.163:0/1828765617 reap_dead start 2024-05-08T06:37:50.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.157+0000 7f76bd578640 1 -- 172.21.15.163:0/1828765617 shutdown_connections 2024-05-08T06:37:50.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.157+0000 7f76bd578640 1 -- 172.21.15.163:0/1828765617 >> 172.21.15.163:0/1828765617 conn(0x7f76b80b3c30 msgr2=0x7f76b80b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:50.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.157+0000 7f76bd578640 1 -- 172.21.15.163:0/1828765617 shutdown_connections 2024-05-08T06:37:50.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.157+0000 7f76bd578640 1 -- 172.21.15.163:0/1828765617 wait complete. 2024-05-08T06:37:50.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.158+0000 7f76bd578640 1 Processor -- start 2024-05-08T06:37:50.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.158+0000 7f76bd578640 1 -- start start 2024-05-08T06:37:50.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76bd578640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a4004800 0x7f76b806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76bd578640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b8157950 0x7f76b806fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76bd578640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76b8157e80 0x7f76b81f0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76bd578640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f76b806f440 con 0x7f76a4004800 2024-05-08T06:37:50.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76bd578640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f76b815e4f0 con 0x7f76b8157950 2024-05-08T06:37:50.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76bd578640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f76b80b16a0 con 0x7f76b8157e80 2024-05-08T06:37:50.167 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76b6ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a4004800 0x7f76b806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.167 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76b6ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a4004800 0x7f76b806f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35136/0 (socket says 172.21.15.163:35136) 2024-05-08T06:37:50.167 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76b6ffd640 1 -- 172.21.15.163:0/1591871849 learned_addr learned my addr 172.21.15.163:0/1591871849 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:50.167 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76ae7fc640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b8157950 0x7f76b806fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.167 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76b77fe640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76b8157e80 0x7f76b81f0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.167 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76b6ffd640 1 -- 172.21.15.163:0/1591871849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76b8157e80 msgr2=0x7f76b81f0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.167 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76b6ffd640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76b8157e80 0x7f76b81f0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.167 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76b6ffd640 1 -- 172.21.15.163:0/1591871849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b8157950 msgr2=0x7f76b806fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76b6ffd640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b8157950 0x7f76b806fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76b6ffd640 1 -- 172.21.15.163:0/1591871849 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b80b63d0 con 0x7f76a4004800 2024-05-08T06:37:50.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76b77fe640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76b8157e80 0x7f76b81f0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:50.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76b6ffd640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a4004800 0x7f76b806f5c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f76a800f820 tx=0x7f76a800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76affff640 1 -- 172.21.15.163:0/1591871849 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a80050d0 con 0x7f76a4004800 2024-05-08T06:37:50.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76affff640 1 -- 172.21.15.163:0/1591871849 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a8018070 con 0x7f76a4004800 2024-05-08T06:37:50.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.159+0000 7f76bd578640 1 -- 172.21.15.163:0/1591871849 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b80b66b0 con 0x7f76a4004800 2024-05-08T06:37:50.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.160+0000 7f76affff640 1 -- 172.21.15.163:0/1591871849 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8005490 con 0x7f76a4004800 2024-05-08T06:37:50.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.160+0000 7f76bd578640 1 -- 172.21.15.163:0/1591871849 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b80b6c70 con 0x7f76a4004800 2024-05-08T06:37:50.169 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.161+0000 7f76bd578640 1 -- 172.21.15.163:0/1591871849 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b806a4b0 con 0x7f76a4004800 2024-05-08T06:37:50.169 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.161+0000 7f76affff640 1 -- 172.21.15.163:0/1591871849 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f76a800d730 con 0x7f76a4004800 2024-05-08T06:37:50.169 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.161+0000 7f76affff640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7698076300 0x7f76980787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.170 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.162+0000 7f76ae7fc640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7698076300 0x7f76980787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.170 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.162+0000 7f76affff640 1 -- 172.21.15.163:0/1591871849 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f76a80970b0 con 0x7f76a4004800 2024-05-08T06:37:50.170 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.162+0000 7f76ae7fc640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7698076300 0x7f76980787c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f76a0033000 tx=0x7f76a0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.170 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.166+0000 7f76affff640 1 -- 172.21.15.163:0/1591871849 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f76a8060d40 con 0x7f76a4004800 2024-05-08T06:37:50.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.346+0000 7f76bd578640 1 -- 172.21.15.163:0/1591871849 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f76b8074640 con 0x7f7698076300 2024-05-08T06:37:50.346 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.346+0000 7f76affff640 1 -- 172.21.15.163:0/1591871849 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f76b8074640 con 0x7f7698076300 2024-05-08T06:37:50.346 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 6 to deep-scrub 2024-05-08T06:37:50.348 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.349+0000 7f76bd578640 1 -- 172.21.15.163:0/1591871849 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7698076300 msgr2=0x7f76980787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.348 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.349+0000 7f76bd578640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7698076300 0x7f76980787c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f76a0033000 tx=0x7f76a0008030 comp rx=0 tx=0).stop 2024-05-08T06:37:50.348 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.349+0000 7f76bd578640 1 -- 172.21.15.163:0/1591871849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a4004800 msgr2=0x7f76b806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.348 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.349+0000 7f76bd578640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a4004800 0x7f76b806f5c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f76a800f820 tx=0x7f76a800fce0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.349+0000 7f76bd578640 1 -- 172.21.15.163:0/1591871849 shutdown_connections 2024-05-08T06:37:50.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.349+0000 7f76bd578640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f7698076300 0x7f76980787c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.349+0000 7f76bd578640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76b8157e80 0x7f76b81f0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.349+0000 7f76bd578640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76b8157950 0x7f76b806fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.349+0000 7f76bd578640 1 --2- 172.21.15.163:0/1591871849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a4004800 0x7f76b806f5c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.349+0000 7f76bd578640 1 -- 172.21.15.163:0/1591871849 >> 172.21.15.163:0/1591871849 conn(0x7f76b80b3c30 msgr2=0x7f76b815f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:50.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.349+0000 7f76bd578640 1 -- 172.21.15.163:0/1591871849 shutdown_connections 2024-05-08T06:37:50.349 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.349+0000 7f76bd578640 1 -- 172.21.15.163:0/1591871849 wait complete. 2024-05-08T06:37:50.359 INFO:tasks.ceph:Scrubbing osd.7 2024-05-08T06:37:50.359 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-08T06:37:50.464 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.464+0000 7f03a6254640 1 Processor -- start 2024-05-08T06:37:50.464 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.465+0000 7f03a6254640 1 -- start start 2024-05-08T06:37:50.465 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.465+0000 7f03a6254640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f03a00d28a0 0x7f03a00b8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.466 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.465+0000 7f03a6254640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f03a00d4810 0x7f03a00b8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.466 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.465+0000 7f03a6254640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03a00b9cf0 con 0x7f03a015f4a0 2024-05-08T06:37:50.466 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.465+0000 7f03a6254640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f03a00d3370 con 0x7f03a00d4810 2024-05-08T06:37:50.466 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.465+0000 7f03a6254640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f03a00b9fd0 con 0x7f03a00d28a0 2024-05-08T06:37:50.466 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.465+0000 7f039ffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f03a00d4810 0x7f03a00b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.466 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.466+0000 7f039ffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f03a00d4810 0x7f03a00b8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35152/0 (socket says 172.21.15.163:35152) 2024-05-08T06:37:50.466 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.466+0000 7f039ffff640 1 -- 172.21.15.163:0/3331268940 learned_addr learned my addr 172.21.15.163:0/3331268940 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:50.466 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.466+0000 7f0396ffd640 1 --2- 172.21.15.163:0/3331268940 >> v2:172.21.15.192:3300/0 conn(0x7f03a00d28a0 0x7f03a00b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.466+0000 7f039ffff640 1 -- 172.21.15.163:0/3331268940 >> v2:172.21.15.192:3300/0 conn(0x7f03a00d28a0 msgr2=0x7f03a00b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:50.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.466+0000 7f039ffff640 1 --2- 172.21.15.163:0/3331268940 >> v2:172.21.15.192:3300/0 conn(0x7f03a00d28a0 0x7f03a00b8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.466+0000 7f039ffff640 1 -- 172.21.15.163:0/3331268940 >> v1:172.21.15.192:6789/0 conn(0x7f03a015f4a0 legacy=0x7f03a00d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:50.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.466+0000 7f039ffff640 1 -- 172.21.15.163:0/3331268940 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a00ba2b0 con 0x7f03a00d4810 2024-05-08T06:37:50.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.466+0000 7f039ffff640 1 --2- 172.21.15.163:0/3331268940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a00d4810 0x7f03a00b8740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f03a00b7480 tx=0x7f039802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f8b308c581b7e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.467+0000 7f039effd640 1 -- 172.21.15.163:0/3331268940 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398036070 con 0x7f03a00d4810 2024-05-08T06:37:50.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.467+0000 7f039effd640 1 -- 172.21.15.163:0/3331268940 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0398031600 con 0x7f03a00d4810 2024-05-08T06:37:50.467 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.467+0000 7f039effd640 1 -- 172.21.15.163:0/3331268940 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03980319d0 con 0x7f03a00d4810 2024-05-08T06:37:50.470 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.467+0000 7f03a6254640 1 -- 172.21.15.163:0/3331268940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a00d4810 msgr2=0x7f03a00b8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.470 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.467+0000 7f03a6254640 1 --2- 172.21.15.163:0/3331268940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a00d4810 0x7f03a00b8740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f03a00b7480 tx=0x7f039802a730 comp rx=0 tx=0).stop 2024-05-08T06:37:50.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.467+0000 7f03a6254640 1 -- 172.21.15.163:0/3331268940 shutdown_connections 2024-05-08T06:37:50.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.467+0000 7f03a6254640 1 --2- 172.21.15.163:0/3331268940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a00d4810 0x7f03a00b8740 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.467+0000 7f03a6254640 1 --2- 172.21.15.163:0/3331268940 >> v2:172.21.15.192:3300/0 conn(0x7f03a00d28a0 0x7f03a00b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.467+0000 7f03a6254640 1 -- 172.21.15.163:0/3331268940 >> 172.21.15.163:0/3331268940 conn(0x7f03a0073e70 msgr2=0x7f03a0074280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:50.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.467+0000 7f03a6254640 1 -- 172.21.15.163:0/3331268940 shutdown_connections 2024-05-08T06:37:50.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.467+0000 7f03a6254640 1 -- 172.21.15.163:0/3331268940 wait complete. 2024-05-08T06:37:50.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.468+0000 7f03a6254640 1 Processor -- start 2024-05-08T06:37:50.471 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.469+0000 7f03a6254640 1 -- start start 2024-05-08T06:37:50.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.469+0000 7f03a6254640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03a00d28a0 0x7f03a0159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.469+0000 7f03a6254640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03a00d4810 0x7f03a01f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.469+0000 7f03a6254640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a015f4a0 0x7f03a0162300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.469+0000 7f03a6254640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f03a00bb080 con 0x7f03a015f4a0 2024-05-08T06:37:50.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.469+0000 7f03a6254640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f03a00baf00 con 0x7f03a00d4810 2024-05-08T06:37:50.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.469+0000 7f03a6254640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f03a00bb200 con 0x7f03a00d28a0 2024-05-08T06:37:50.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f039ffff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a015f4a0 0x7f03a0162300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f039ffff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a015f4a0 0x7f03a0162300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35168/0 (socket says 172.21.15.163:35168) 2024-05-08T06:37:50.472 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f039ffff640 1 -- 172.21.15.163:0/816743212 learned_addr learned my addr 172.21.15.163:0/816743212 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:50.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f0396ffd640 1 --2- 172.21.15.163:0/816743212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03a00d4810 0x7f03a01f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f039f7fe640 1 --2- 172.21.15.163:0/816743212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03a00d28a0 0x7f03a0159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f039ffff640 1 -- 172.21.15.163:0/816743212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03a00d28a0 msgr2=0x7f03a0159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f039ffff640 1 --2- 172.21.15.163:0/816743212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03a00d28a0 0x7f03a0159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f039ffff640 1 -- 172.21.15.163:0/816743212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03a00d4810 msgr2=0x7f03a01f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f039ffff640 1 --2- 172.21.15.163:0/816743212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03a00d4810 0x7f03a01f5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f039ffff640 1 -- 172.21.15.163:0/816743212 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a015f9b0 con 0x7f03a015f4a0 2024-05-08T06:37:50.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f039ffff640 1 --2- 172.21.15.163:0/816743212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a015f4a0 0x7f03a0162300 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0398009f40 tx=0x7f0398004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.473 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f039cff9640 1 -- 172.21.15.163:0/816743212 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398036050 con 0x7f03a015f4a0 2024-05-08T06:37:50.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f039cff9640 1 -- 172.21.15.163:0/816743212 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0398042070 con 0x7f03a015f4a0 2024-05-08T06:37:50.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f039cff9640 1 -- 172.21.15.163:0/816743212 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398009510 con 0x7f03a015f4a0 2024-05-08T06:37:50.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f03a6254640 1 -- 172.21.15.163:0/816743212 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a015fc90 con 0x7f03a015f4a0 2024-05-08T06:37:50.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.470+0000 7f03a6254640 1 -- 172.21.15.163:0/816743212 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a015a140 con 0x7f03a015f4a0 2024-05-08T06:37:50.474 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.471+0000 7f03a6254640 1 -- 172.21.15.163:0/816743212 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03a0160350 con 0x7f03a015f4a0 2024-05-08T06:37:50.475 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.472+0000 7f039cff9640 1 -- 172.21.15.163:0/816743212 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0398007bf0 con 0x7f03a015f4a0 2024-05-08T06:37:50.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.472+0000 7f039cff9640 1 --2- 172.21.15.163:0/816743212 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f03800762d0 0x7f0380078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.473+0000 7f039cff9640 1 -- 172.21.15.163:0/816743212 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f03980b8c70 con 0x7f03a015f4a0 2024-05-08T06:37:50.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.473+0000 7f039f7fe640 1 --2- 172.21.15.163:0/816743212 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f03800762d0 0x7f0380078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.473+0000 7f039cff9640 1 --2- 172.21.15.163:0/816743212 >> [v2:172.21.15.192:6808/1747004047,v1:172.21.15.192:6809/1747004047] conn(0x7f038007c230 0x7f038007e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.473+0000 7f039cff9640 1 -- 172.21.15.163:0/816743212 --> [v2:172.21.15.192:6808/1747004047,v1:172.21.15.192:6809/1747004047] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f038007ed00 con 0x7f038007c230 2024-05-08T06:37:50.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.473+0000 7f039cff9640 1 -- 172.21.15.163:0/816743212 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f03980b9060 con 0x7f03a015f4a0 2024-05-08T06:37:50.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.473+0000 7f0396ffd640 1 --2- 172.21.15.163:0/816743212 >> [v2:172.21.15.192:6808/1747004047,v1:172.21.15.192:6809/1747004047] conn(0x7f038007c230 0x7f038007e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.476 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.473+0000 7f039f7fe640 1 --2- 172.21.15.163:0/816743212 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f03800762d0 0x7f0380078790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0390006fd0 tx=0x7f0390009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.473+0000 7f0396ffd640 1 --2- 172.21.15.163:0/816743212 >> [v2:172.21.15.192:6808/1747004047,v1:172.21.15.192:6809/1747004047] conn(0x7f038007c230 0x7f038007e650 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.477 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.474+0000 7f039cff9640 1 -- 172.21.15.163:0/816743212 <== osd.7 v2:172.21.15.192:6808/1747004047 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f038007ed00 con 0x7f038007c230 2024-05-08T06:37:50.486 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.487+0000 7f03a6254640 1 -- 172.21.15.163:0/816743212 --> [v2:172.21.15.192:6808/1747004047,v1:172.21.15.192:6809/1747004047] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f03a00b4a80 con 0x7f038007c230 2024-05-08T06:37:50.487 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.488+0000 7f039cff9640 1 -- 172.21.15.163:0/816743212 <== osd.7 v2:172.21.15.192:6808/1747004047 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f03a00b4a80 con 0x7f038007c230 2024-05-08T06:37:50.487 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:37:50.487 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T06:37:50.488 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:37:50.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.488+0000 7f03a6254640 1 -- 172.21.15.163:0/816743212 >> [v2:172.21.15.192:6808/1747004047,v1:172.21.15.192:6809/1747004047] conn(0x7f038007c230 msgr2=0x7f038007e650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.488+0000 7f03a6254640 1 --2- 172.21.15.163:0/816743212 >> [v2:172.21.15.192:6808/1747004047,v1:172.21.15.192:6809/1747004047] conn(0x7f038007c230 0x7f038007e650 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.488+0000 7f03a6254640 1 -- 172.21.15.163:0/816743212 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f03800762d0 msgr2=0x7f0380078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.488+0000 7f03a6254640 1 --2- 172.21.15.163:0/816743212 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f03800762d0 0x7f0380078790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0390006fd0 tx=0x7f0390009060 comp rx=0 tx=0).stop 2024-05-08T06:37:50.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.488+0000 7f03a6254640 1 -- 172.21.15.163:0/816743212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a015f4a0 msgr2=0x7f03a0162300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.488+0000 7f03a6254640 1 --2- 172.21.15.163:0/816743212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03a015f4a0 0x7f03a0162300 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0398009f40 tx=0x7f0398004a40 comp rx=0 tx=0).stop 2024-05-08T06:37:50.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.488+0000 7f039ffff640 1 -- 172.21.15.163:0/816743212 reap_dead start 2024-05-08T06:37:50.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.488+0000 7f03a6254640 1 -- 172.21.15.163:0/816743212 shutdown_connections 2024-05-08T06:37:50.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.488+0000 7f03a6254640 1 -- 172.21.15.163:0/816743212 >> 172.21.15.163:0/816743212 conn(0x7f03a0073e70 msgr2=0x7f03a00b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:50.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.489+0000 7f03a6254640 1 -- 172.21.15.163:0/816743212 shutdown_connections 2024-05-08T06:37:50.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.489+0000 7f03a6254640 1 -- 172.21.15.163:0/816743212 wait complete. 2024-05-08T06:37:50.498 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-08T06:37:50.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.600+0000 7f4600859640 1 Processor -- start 2024-05-08T06:37:50.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.600+0000 7f4600859640 1 -- start start 2024-05-08T06:37:50.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.601+0000 7f4600859640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f45fc0d1260 0x7f45fc0d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.601+0000 7f4600859640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f45fc0b6160 0x7f45fc15ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.601+0000 7f4600859640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45fc0d1b80 con 0x7f45fc15d3d0 2024-05-08T06:37:50.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.601+0000 7f4600859640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f45fc0aff30 con 0x7f45fc0d1260 2024-05-08T06:37:50.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.601+0000 7f4600859640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f45fc0b0210 con 0x7f45fc0b6160 2024-05-08T06:37:50.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.601+0000 7f45fa59c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f45fc0d1260 0x7f45fc0d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.601+0000 7f45f3fff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f45fc15d3d0 0x7f45fc06e380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:35344/0 (socket says 172.21.15.163:35344) 2024-05-08T06:37:50.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.601+0000 7f45f3fff640 1 -- 172.21.15.163:0/4030174640 learned_addr learned my addr 172.21.15.163:0/4030174640 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:50.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.601+0000 7f45fad9d640 1 --2- 172.21.15.163:0/4030174640 >> v2:172.21.15.192:3300/0 conn(0x7f45fc0b6160 0x7f45fc15ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.601+0000 7f45fa59c640 1 -- 172.21.15.163:0/4030174640 >> v2:172.21.15.192:3300/0 conn(0x7f45fc0b6160 msgr2=0x7f45fc15ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:37:50.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.601+0000 7f45fa59c640 1 --2- 172.21.15.163:0/4030174640 >> v2:172.21.15.192:3300/0 conn(0x7f45fc0b6160 0x7f45fc15ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.603 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.601+0000 7f45fa59c640 1 -- 172.21.15.163:0/4030174640 >> v1:172.21.15.163:6789/0 conn(0x7f45fc15d3d0 legacy=0x7f45fc06e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.601+0000 7f45f9d9b640 1 -- 172.21.15.163:0/4030174640 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3679713577 0 0) 0x7f45fc0d1b80 con 0x7f45fc15d3d0 2024-05-08T06:37:50.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.601+0000 7f45fa59c640 1 -- 172.21.15.163:0/4030174640 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45fc0b04f0 con 0x7f45fc0d1260 2024-05-08T06:37:50.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.602+0000 7f45fa59c640 1 --2- 172.21.15.163:0/4030174640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45fc0d1260 0x7f45fc0d1640 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0b6090 tx=0x7f45ec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6624b70e50717e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.602+0000 7f45f9d9b640 1 -- 172.21.15.163:0/4030174640 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec037070 con 0x7f45fc0d1260 2024-05-08T06:37:50.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.602+0000 7f45f9d9b640 1 -- 172.21.15.163:0/4030174640 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45ec032440 con 0x7f45fc0d1260 2024-05-08T06:37:50.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.602+0000 7f45f9d9b640 1 -- 172.21.15.163:0/4030174640 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec032810 con 0x7f45fc0d1260 2024-05-08T06:37:50.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.603+0000 7f4600859640 1 -- 172.21.15.163:0/4030174640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45fc0d1260 msgr2=0x7f45fc0d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.604 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.603+0000 7f4600859640 1 --2- 172.21.15.163:0/4030174640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45fc0d1260 0x7f45fc0d1640 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0b6090 tx=0x7f45ec02a1a0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.603+0000 7f4600859640 1 -- 172.21.15.163:0/4030174640 shutdown_connections 2024-05-08T06:37:50.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.603+0000 7f4600859640 1 --2- 172.21.15.163:0/4030174640 >> v2:172.21.15.192:3300/0 conn(0x7f45fc0b6160 0x7f45fc15ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.603+0000 7f4600859640 1 --2- 172.21.15.163:0/4030174640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45fc0d1260 0x7f45fc0d1640 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.603+0000 7f4600859640 1 -- 172.21.15.163:0/4030174640 >> 172.21.15.163:0/4030174640 conn(0x7f45fc06c740 msgr2=0x7f45fc06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:50.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.603+0000 7f4600859640 1 -- 172.21.15.163:0/4030174640 shutdown_connections 2024-05-08T06:37:50.605 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.603+0000 7f4600859640 1 -- 172.21.15.163:0/4030174640 wait complete. 2024-05-08T06:37:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.604+0000 7f4600859640 1 Processor -- start 2024-05-08T06:37:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.604+0000 7f4600859640 1 -- start start 2024-05-08T06:37:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f4600859640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45fc0b6160 0x7f45fc0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f4600859640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45fc0d1260 0x7f45fc0c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f4600859640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45fc15d3d0 0x7f45fc0c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f4600859640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc06ea90 con 0x7f45fc0b6160 2024-05-08T06:37:50.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f4600859640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc15d8b0 con 0x7f45fc0d1260 2024-05-08T06:37:50.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f4600859640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f45fc0afb50 con 0x7f45fc15d3d0 2024-05-08T06:37:50.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f45fa59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45fc0b6160 0x7f45fc0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f45fa59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45fc0b6160 0x7f45fc0ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35192/0 (socket says 172.21.15.163:35192) 2024-05-08T06:37:50.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f45fa59c640 1 -- 172.21.15.163:0/2201970070 learned_addr learned my addr 172.21.15.163:0/2201970070 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:50.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f45f3fff640 1 --2- 172.21.15.163:0/2201970070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45fc0d1260 0x7f45fc0c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f45fa59c640 1 -- 172.21.15.163:0/2201970070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45fc15d3d0 msgr2=0x7f45fc0c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f45fa59c640 1 --2- 172.21.15.163:0/2201970070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45fc15d3d0 0x7f45fc0c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f45fa59c640 1 -- 172.21.15.163:0/2201970070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45fc0d1260 msgr2=0x7f45fc0c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f45fa59c640 1 --2- 172.21.15.163:0/2201970070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45fc0d1260 0x7f45fc0c5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f45fa59c640 1 -- 172.21.15.163:0/2201970070 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45fc1f7ae0 con 0x7f45fc0b6160 2024-05-08T06:37:50.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f45fa59c640 1 --2- 172.21.15.163:0/2201970070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45fc0b6160 0x7f45fc0ce0d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f45ec000c00 tx=0x7f45ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f45f2ffd640 1 -- 172.21.15.163:0/2201970070 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec037050 con 0x7f45fc0b6160 2024-05-08T06:37:50.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f45f2ffd640 1 -- 172.21.15.163:0/2201970070 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45ec038040 con 0x7f45fc0b6160 2024-05-08T06:37:50.610 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.605+0000 7f4600859640 1 -- 172.21.15.163:0/2201970070 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45fc1f6d20 con 0x7f45fc0b6160 2024-05-08T06:37:50.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.606+0000 7f45f2ffd640 1 -- 172.21.15.163:0/2201970070 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec006e10 con 0x7f45fc0b6160 2024-05-08T06:37:50.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.606+0000 7f4600859640 1 -- 172.21.15.163:0/2201970070 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45fc1f72e0 con 0x7f45fc0b6160 2024-05-08T06:37:50.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.607+0000 7f4600859640 1 -- 172.21.15.163:0/2201970070 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45fc06a4b0 con 0x7f45fc0b6160 2024-05-08T06:37:50.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.607+0000 7f45f2ffd640 1 -- 172.21.15.163:0/2201970070 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f45ec030980 con 0x7f45fc0b6160 2024-05-08T06:37:50.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.607+0000 7f45f2ffd640 1 --2- 172.21.15.163:0/2201970070 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f45dc0762d0 0x7f45dc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.608+0000 7f45f2ffd640 1 -- 172.21.15.163:0/2201970070 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f45ec0b7e70 con 0x7f45fc0b6160 2024-05-08T06:37:50.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.608+0000 7f45f3fff640 1 --2- 172.21.15.163:0/2201970070 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f45dc0762d0 0x7f45dc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.611 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.608+0000 7f45f3fff640 1 --2- 172.21.15.163:0/2201970070 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f45dc0762d0 0x7f45dc078790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f45fc09ea40 tx=0x7f45e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.612+0000 7f45f2ffd640 1 -- 172.21.15.163:0/2201970070 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f45ec081a70 con 0x7f45fc0b6160 2024-05-08T06:37:50.793 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.793+0000 7f4600859640 1 -- 172.21.15.163:0/2201970070 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f45fc0b54a0 con 0x7f45dc0762d0 2024-05-08T06:37:50.793 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.794+0000 7f45f2ffd640 1 -- 172.21.15.163:0/2201970070 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f45fc0b54a0 con 0x7f45dc0762d0 2024-05-08T06:37:50.793 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 7 to deep-scrub 2024-05-08T06:37:50.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.796+0000 7f4600859640 1 -- 172.21.15.163:0/2201970070 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f45dc0762d0 msgr2=0x7f45dc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.796+0000 7f4600859640 1 --2- 172.21.15.163:0/2201970070 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f45dc0762d0 0x7f45dc078790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f45fc09ea40 tx=0x7f45e4009040 comp rx=0 tx=0).stop 2024-05-08T06:37:50.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.796+0000 7f4600859640 1 -- 172.21.15.163:0/2201970070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45fc0b6160 msgr2=0x7f45fc0ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.796+0000 7f4600859640 1 --2- 172.21.15.163:0/2201970070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45fc0b6160 0x7f45fc0ce0d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f45ec000c00 tx=0x7f45ec0046c0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.796+0000 7f4600859640 1 -- 172.21.15.163:0/2201970070 shutdown_connections 2024-05-08T06:37:50.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.796+0000 7f4600859640 1 --2- 172.21.15.163:0/2201970070 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f45dc0762d0 0x7f45dc078790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.796+0000 7f4600859640 1 --2- 172.21.15.163:0/2201970070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45fc15d3d0 0x7f45fc0c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.796+0000 7f4600859640 1 --2- 172.21.15.163:0/2201970070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45fc0d1260 0x7f45fc0c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.796+0000 7f4600859640 1 --2- 172.21.15.163:0/2201970070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45fc0b6160 0x7f45fc0ce0d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.796+0000 7f4600859640 1 -- 172.21.15.163:0/2201970070 >> 172.21.15.163:0/2201970070 conn(0x7f45fc06c740 msgr2=0x7f45fc0cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:50.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.796+0000 7f4600859640 1 -- 172.21.15.163:0/2201970070 shutdown_connections 2024-05-08T06:37:50.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.796+0000 7f4600859640 1 -- 172.21.15.163:0/2201970070 wait complete. 2024-05-08T06:37:50.806 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:37:50.906 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.907+0000 7f8ee2647640 1 Processor -- start 2024-05-08T06:37:50.906 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.907+0000 7f8ee2647640 1 -- start start 2024-05-08T06:37:50.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.908+0000 7f8ee2647640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8edc0ae180 0x7f8edc0b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.908+0000 7f8ee2647640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8edc072f80 con 0x7f8edc0b46e0 2024-05-08T06:37:50.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.908+0000 7f8ee2647640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8edc074430 con 0x7f8edc15c020 2024-05-08T06:37:50.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.908+0000 7f8ee2647640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f8edc0ae700 con 0x7f8edc0ae180 2024-05-08T06:37:50.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.908+0000 7f8edb7fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8edc0ae180 0x7f8edc0b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.908+0000 7f8ee0be4640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8edc0b46e0 0x7f8edc0afc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:35352/0 (socket says 172.21.15.163:35352) 2024-05-08T06:37:50.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.908+0000 7f8ee0be4640 1 -- 172.21.15.163:0/2301181124 learned_addr learned my addr 172.21.15.163:0/2301181124 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:37:50.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.908+0000 7f8edb7fe640 1 -- 172.21.15.163:0/2301181124 >> v1:172.21.15.192:6789/0 conn(0x7f8edc15c020 legacy=0x7f8edc073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.908+0000 7f8edb7fe640 1 -- 172.21.15.163:0/2301181124 >> v1:172.21.15.163:6789/0 conn(0x7f8edc0b46e0 legacy=0x7f8edc0afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.908+0000 7f8edb7fe640 1 -- 172.21.15.163:0/2301181124 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8edc074710 con 0x7f8edc0ae180 2024-05-08T06:37:50.910 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.909+0000 7f8edb7fe640 1 --2- 172.21.15.163:0/2301181124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0ae180 0x7f8edc0b41a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00097a0 tx=0x7f8ed002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab0b12981bed2e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.910 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.909+0000 7f8edaffd640 1 -- 172.21.15.163:0/2301181124 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0037070 con 0x7f8edc0ae180 2024-05-08T06:37:50.910 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.909+0000 7f8edaffd640 1 -- 172.21.15.163:0/2301181124 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed0032440 con 0x7f8edc0ae180 2024-05-08T06:37:50.910 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.909+0000 7f8edaffd640 1 -- 172.21.15.163:0/2301181124 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed002ae60 con 0x7f8edc0ae180 2024-05-08T06:37:50.910 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.910+0000 7f8ee2647640 1 -- 172.21.15.163:0/2301181124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0ae180 msgr2=0x7f8edc0b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.911 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.910+0000 7f8ee2647640 1 --2- 172.21.15.163:0/2301181124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0ae180 0x7f8edc0b41a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00097a0 tx=0x7f8ed002a110 comp rx=0 tx=0).stop 2024-05-08T06:37:50.911 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.910+0000 7f8ee2647640 1 -- 172.21.15.163:0/2301181124 shutdown_connections 2024-05-08T06:37:50.911 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.910+0000 7f8ee2647640 1 --2- 172.21.15.163:0/2301181124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0ae180 0x7f8edc0b41a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.911 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.910+0000 7f8ee2647640 1 -- 172.21.15.163:0/2301181124 >> 172.21.15.163:0/2301181124 conn(0x7f8edc06c500 msgr2=0x7f8edc06c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:50.911 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.910+0000 7f8ee2647640 1 -- 172.21.15.163:0/2301181124 shutdown_connections 2024-05-08T06:37:50.911 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.910+0000 7f8ee2647640 1 -- 172.21.15.163:0/2301181124 wait complete. 2024-05-08T06:37:50.911 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.911+0000 7f8ee2647640 1 Processor -- start 2024-05-08T06:37:50.911 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.911+0000 7f8ee2647640 1 -- start start 2024-05-08T06:37:50.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8ee2647640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0ae180 0x7f8edc152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8ee2647640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8edc0b46e0 0x7f8edc1f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8ee2647640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8edc15c020 0x7f8edc06f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.914 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8ee2647640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc15c400 con 0x7f8edc0ae180 2024-05-08T06:37:50.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8ee2647640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc073290 con 0x7f8edc0b46e0 2024-05-08T06:37:50.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8ee2647640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8edc0b4b70 con 0x7f8edc15c020 2024-05-08T06:37:50.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8edbfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0ae180 0x7f8edc152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8edbfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0ae180 0x7f8edc152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35204/0 (socket says 172.21.15.163:35204) 2024-05-08T06:37:50.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8edbfff640 1 -- 172.21.15.163:0/253270079 learned_addr learned my addr 172.21.15.163:0/253270079 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:37:50.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8edb7fe640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8edc0b46e0 0x7f8edc1f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8ee0be4640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8edc15c020 0x7f8edc06f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8edbfff640 1 -- 172.21.15.163:0/253270079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8edc15c020 msgr2=0x7f8edc06f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8edbfff640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8edc15c020 0x7f8edc06f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8edbfff640 1 -- 172.21.15.163:0/253270079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8edc0b46e0 msgr2=0x7f8edc1f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:50.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8edbfff640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8edc0b46e0 0x7f8edc1f5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:50.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8edbfff640 1 -- 172.21.15.163:0/253270079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8edc152760 con 0x7f8edc0ae180 2024-05-08T06:37:50.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8edb7fe640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8edc0b46e0 0x7f8edc1f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:37:50.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.912+0000 7f8edbfff640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0ae180 0x7f8edc152010 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00c3d0 tx=0x7f8ecc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.913+0000 7f8ed8ff9640 1 -- 172.21.15.163:0/253270079 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc017070 con 0x7f8edc0ae180 2024-05-08T06:37:50.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.913+0000 7f8ed8ff9640 1 -- 172.21.15.163:0/253270079 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ecc0052d0 con 0x7f8edc0ae180 2024-05-08T06:37:50.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.913+0000 7f8ee2647640 1 -- 172.21.15.163:0/253270079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8edc152a40 con 0x7f8edc0ae180 2024-05-08T06:37:50.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.913+0000 7f8ed8ff9640 1 -- 172.21.15.163:0/253270079 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc003830 con 0x7f8edc0ae180 2024-05-08T06:37:50.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.913+0000 7f8ee2647640 1 -- 172.21.15.163:0/253270079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8edc153000 con 0x7f8edc0ae180 2024-05-08T06:37:50.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.914+0000 7f8ee2647640 1 -- 172.21.15.163:0/253270079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8edc06a550 con 0x7f8edc0ae180 2024-05-08T06:37:50.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.914+0000 7f8ed8ff9640 1 -- 172.21.15.163:0/253270079 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8ecc01a070 con 0x7f8edc0ae180 2024-05-08T06:37:50.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.915+0000 7f8ed8ff9640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8ec80762d0 0x7f8ec8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:37:50.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.915+0000 7f8edb7fe640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8ec80762d0 0x7f8ec8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:37:50.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.915+0000 7f8ed8ff9640 1 -- 172.21.15.163:0/253270079 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f8ecc096630 con 0x7f8edc0ae180 2024-05-08T06:37:50.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.915+0000 7f8edb7fe640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8ec80762d0 0x7f8ec8078790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0034000 tx=0x7f8ed002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:37:50.918 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:50.919+0000 7f8ed8ff9640 1 -- 172.21.15.163:0/253270079 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8ecc0602c0 con 0x7f8edc0ae180 2024-05-08T06:37:51.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:51.095+0000 7f8ee2647640 1 -- 172.21.15.163:0/253270079 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8edc15f230 con 0x7f8ec80762d0 2024-05-08T06:37:51.094 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:51.096+0000 7f8ed8ff9640 1 -- 172.21.15.163:0/253270079 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13340 (secure 0 0 0) 0x7f8edc15f230 con 0x7f8ec80762d0 2024-05-08T06:37:51.095 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:37:51.095 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2024-05-08T06:37:51.097 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:51.098+0000 7f8ee2647640 1 -- 172.21.15.163:0/253270079 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8ec80762d0 msgr2=0x7f8ec8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:51.097 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:51.098+0000 7f8ee2647640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8ec80762d0 0x7f8ec8078790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0034000 tx=0x7f8ed002e000 comp rx=0 tx=0).stop 2024-05-08T06:37:51.097 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:51.098+0000 7f8ee2647640 1 -- 172.21.15.163:0/253270079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0ae180 msgr2=0x7f8edc152010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:37:51.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:51.098+0000 7f8ee2647640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0ae180 0x7f8edc152010 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00c3d0 tx=0x7f8ecc00c890 comp rx=0 tx=0).stop 2024-05-08T06:37:51.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:51.098+0000 7f8ee2647640 1 -- 172.21.15.163:0/253270079 shutdown_connections 2024-05-08T06:37:51.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:51.098+0000 7f8ee2647640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f8ec80762d0 0x7f8ec8078790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:51.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:51.098+0000 7f8ee2647640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8edc15c020 0x7f8edc06f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:51.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:51.098+0000 7f8ee2647640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8edc0b46e0 0x7f8edc1f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:51.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:51.098+0000 7f8ee2647640 1 --2- 172.21.15.163:0/253270079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0ae180 0x7f8edc152010 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:37:51.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:51.098+0000 7f8ee2647640 1 -- 172.21.15.163:0/253270079 >> 172.21.15.163:0/253270079 conn(0x7f8edc06c500 msgr2=0x7f8edc0afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:37:51.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:51.098+0000 7f8ee2647640 1 -- 172.21.15.163:0/253270079 shutdown_connections 2024-05-08T06:37:51.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:37:51.098+0000 7f8ee2647640 1 -- 172.21.15.163:0/253270079 wait complete. 2024-05-08T06:37:51.108 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-05-08T06:37:50.110773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220488,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754232,"statfs":{"total":773094113280,"available":772868333568,"internally_reserved":0,"allocated":4431872,"data_stored":2445648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.329393"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":138,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-08T06:37:43.319549+0000","last_change":"2024-05-08T06:36:31.710287+0000","last_active":"2024-05-08T06:37:43.319549+0000","last_peered":"2024-05-08T06:37:43.319549+0000","last_clean":"2024-05-08T06:37:43.319549+0000","last_became_active":"2024-05-08T06:36:31.709270+0000","last_became_peered":"2024-05-08T06:36:31.709270+0000","last_unstale":"2024-05-08T06:37:43.319549+0000","last_undegraded":"2024-05-08T06:37:43.319549+0000","last_fullsized":"2024-05-08T06:37:43.319549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:36:30.122918+0000","last_clean_scrub_stamp":"2024-05-08T06:36:30.122918+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:42.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":999424,"data_stored":748482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":999424,"data_stored":748482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:37:51.110 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T06:36:30.122918+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=6, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=6, tm_min=37, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T06:37:51.110 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T06:38:11.111 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:38:11.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.225+0000 7fa764a08640 1 Processor -- start 2024-05-08T06:38:11.225 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.225+0000 7fa764a08640 1 -- start start 2024-05-08T06:38:11.226 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.226+0000 7fa764a08640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa76015f760 0x7fa7600b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.226 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.226+0000 7fa764a08640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa7600d0a00 0x7fa7600b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.226 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.226+0000 7fa764a08640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7600d0070 con 0x7fa7600cfc10 2024-05-08T06:38:11.226 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.226+0000 7fa764a08640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa76015fb40 con 0x7fa76015f760 2024-05-08T06:38:11.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.226+0000 7fa764a08640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa7600d36f0 con 0x7fa7600d0a00 2024-05-08T06:38:11.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.226+0000 7fa75e59c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa76015f760 0x7fa7600b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.226+0000 7fa755d9b640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa7600cfc10 0x7fa7600d02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:55506/0 (socket says 172.21.15.163:55506) 2024-05-08T06:38:11.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.226+0000 7fa755d9b640 1 -- 172.21.15.163:0/697499239 learned_addr learned my addr 172.21.15.163:0/697499239 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:38:11.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.227+0000 7fa75ed9d640 1 --2- 172.21.15.163:0/697499239 >> v2:172.21.15.192:3300/0 conn(0x7fa7600d0a00 0x7fa7600b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.227+0000 7fa75e59c640 1 -- 172.21.15.163:0/697499239 >> v2:172.21.15.192:3300/0 conn(0x7fa7600d0a00 msgr2=0x7fa7600b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:38:11.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.227+0000 7fa75e59c640 1 --2- 172.21.15.163:0/697499239 >> v2:172.21.15.192:3300/0 conn(0x7fa7600d0a00 0x7fa7600b8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.227+0000 7fa75e59c640 1 -- 172.21.15.163:0/697499239 >> v1:172.21.15.163:6789/0 conn(0x7fa7600cfc10 legacy=0x7fa7600d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.227+0000 7fa75e59c640 1 -- 172.21.15.163:0/697499239 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7600d39d0 con 0x7fa76015f760 2024-05-08T06:38:11.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.227+0000 7fa75e59c640 1 --2- 172.21.15.163:0/697499239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76015f760 0x7fa7600b4770 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa76006dfa0 tx=0x7fa75002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adb408eb56a9602 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:38:11.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.228+0000 7fa75dd9b640 1 -- 172.21.15.163:0/697499239 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750037070 con 0x7fa76015f760 2024-05-08T06:38:11.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.228+0000 7fa75dd9b640 1 -- 172.21.15.163:0/697499239 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa750032440 con 0x7fa76015f760 2024-05-08T06:38:11.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.228+0000 7fa75dd9b640 1 -- 172.21.15.163:0/697499239 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75002ae60 con 0x7fa76015f760 2024-05-08T06:38:11.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.228+0000 7fa764a08640 1 -- 172.21.15.163:0/697499239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76015f760 msgr2=0x7fa7600b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.228+0000 7fa764a08640 1 --2- 172.21.15.163:0/697499239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76015f760 0x7fa7600b4770 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa76006dfa0 tx=0x7fa75002a110 comp rx=0 tx=0).stop 2024-05-08T06:38:11.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.228+0000 7fa764a08640 1 -- 172.21.15.163:0/697499239 shutdown_connections 2024-05-08T06:38:11.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.228+0000 7fa764a08640 1 --2- 172.21.15.163:0/697499239 >> v2:172.21.15.192:3300/0 conn(0x7fa7600d0a00 0x7fa7600b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.228+0000 7fa764a08640 1 --2- 172.21.15.163:0/697499239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa76015f760 0x7fa7600b4770 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.228+0000 7fa764a08640 1 -- 172.21.15.163:0/697499239 >> 172.21.15.163:0/697499239 conn(0x7fa7600740e0 msgr2=0x7fa76006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:38:11.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.228+0000 7fa764a08640 1 -- 172.21.15.163:0/697499239 shutdown_connections 2024-05-08T06:38:11.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.228+0000 7fa764a08640 1 -- 172.21.15.163:0/697499239 wait complete. 2024-05-08T06:38:11.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.230+0000 7fa764a08640 1 Processor -- start 2024-05-08T06:38:11.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.230+0000 7fa764a08640 1 -- start start 2024-05-08T06:38:11.233 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa764a08640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7600cfc10 0x7fa760159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.233 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa764a08640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7600d0a00 0x7fa760147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.233 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa764a08640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa76015f760 0x7fa7601482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.233 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa764a08640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa7600d4270 con 0x7fa7600cfc10 2024-05-08T06:38:11.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa764a08640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7600d3f90 con 0x7fa76015f760 2024-05-08T06:38:11.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa764a08640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa7600d4790 con 0x7fa7600d0a00 2024-05-08T06:38:11.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa75e59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7600cfc10 0x7fa760159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa75e59c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7600cfc10 0x7fa760159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40694/0 (socket says 172.21.15.163:40694) 2024-05-08T06:38:11.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa75e59c640 1 -- 172.21.15.163:0/3581169072 learned_addr learned my addr 172.21.15.163:0/3581169072 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:38:11.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa75ed9d640 1 --2- 172.21.15.163:0/3581169072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa76015f760 0x7fa7601482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa75e59c640 1 -- 172.21.15.163:0/3581169072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7600d0a00 msgr2=0x7fa760147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa75e59c640 1 --2- 172.21.15.163:0/3581169072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7600d0a00 0x7fa760147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa75e59c640 1 -- 172.21.15.163:0/3581169072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa76015f760 msgr2=0x7fa7601482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa75e59c640 1 --2- 172.21.15.163:0/3581169072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa76015f760 0x7fa7601482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa75e59c640 1 -- 172.21.15.163:0/3581169072 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76015a240 con 0x7fa7600cfc10 2024-05-08T06:38:11.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.231+0000 7fa75e59c640 1 --2- 172.21.15.163:0/3581169072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7600cfc10 0x7fa760159b10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa750000c00 tx=0x7fa7500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:38:11.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.232+0000 7fa7577fe640 1 -- 172.21.15.163:0/3581169072 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750037040 con 0x7fa7600cfc10 2024-05-08T06:38:11.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.232+0000 7fa7577fe640 1 -- 172.21.15.163:0/3581169072 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa750042070 con 0x7fa7600cfc10 2024-05-08T06:38:11.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.232+0000 7fa764a08640 1 -- 172.21.15.163:0/3581169072 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76015a520 con 0x7fa7600cfc10 2024-05-08T06:38:11.235 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.232+0000 7fa7577fe640 1 -- 172.21.15.163:0/3581169072 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750030bb0 con 0x7fa7600cfc10 2024-05-08T06:38:11.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.232+0000 7fa764a08640 1 -- 172.21.15.163:0/3581169072 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76015aae0 con 0x7fa7600cfc10 2024-05-08T06:38:11.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.233+0000 7fa764a08640 1 -- 172.21.15.163:0/3581169072 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa76006a550 con 0x7fa7600cfc10 2024-05-08T06:38:11.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.233+0000 7fa7577fe640 1 -- 172.21.15.163:0/3581169072 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa750047460 con 0x7fa7600cfc10 2024-05-08T06:38:11.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.234+0000 7fa7577fe640 1 --2- 172.21.15.163:0/3581169072 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa7400762d0 0x7fa740078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.234+0000 7fa7577fe640 1 -- 172.21.15.163:0/3581169072 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fa7500b7fb0 con 0x7fa7600cfc10 2024-05-08T06:38:11.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.234+0000 7fa755d9b640 1 --2- 172.21.15.163:0/3581169072 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa7400762d0 0x7fa740078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.236 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.234+0000 7fa755d9b640 1 --2- 172.21.15.163:0/3581169072 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa7400762d0 0x7fa740078790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa748005880 tx=0x7fa748007a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:38:11.237 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.238+0000 7fa7577fe640 1 -- 172.21.15.163:0/3581169072 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa750032e20 con 0x7fa7600cfc10 2024-05-08T06:38:11.414 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.415+0000 7fa764a08640 1 -- 172.21.15.163:0/3581169072 --> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa76006c320 con 0x7fa7400762d0 2024-05-08T06:38:11.415 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.416+0000 7fa7577fe640 1 -- 172.21.15.163:0/3581169072 <== mgr.4104 v2:172.21.15.163:6832/2781435851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13339 (secure 0 0 0) 0x7fa76006c320 con 0x7fa7400762d0 2024-05-08T06:38:11.415 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:38:11.416 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2024-05-08T06:38:11.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.418+0000 7fa764a08640 1 -- 172.21.15.163:0/3581169072 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa7400762d0 msgr2=0x7fa740078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.418+0000 7fa764a08640 1 --2- 172.21.15.163:0/3581169072 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa7400762d0 0x7fa740078790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa748005880 tx=0x7fa748007a20 comp rx=0 tx=0).stop 2024-05-08T06:38:11.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.418+0000 7fa764a08640 1 -- 172.21.15.163:0/3581169072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7600cfc10 msgr2=0x7fa760159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.418+0000 7fa764a08640 1 --2- 172.21.15.163:0/3581169072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7600cfc10 0x7fa760159b10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa750000c00 tx=0x7fa7500046c0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.418+0000 7fa764a08640 1 -- 172.21.15.163:0/3581169072 shutdown_connections 2024-05-08T06:38:11.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.418+0000 7fa764a08640 1 --2- 172.21.15.163:0/3581169072 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fa7400762d0 0x7fa740078790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.418+0000 7fa764a08640 1 --2- 172.21.15.163:0/3581169072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa76015f760 0x7fa7601482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.419 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.418+0000 7fa764a08640 1 --2- 172.21.15.163:0/3581169072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7600d0a00 0x7fa760147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.419 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.418+0000 7fa764a08640 1 --2- 172.21.15.163:0/3581169072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7600cfc10 0x7fa760159b10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.419 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.418+0000 7fa764a08640 1 -- 172.21.15.163:0/3581169072 >> 172.21.15.163:0/3581169072 conn(0x7fa7600740e0 msgr2=0x7fa7600c91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:38:11.419 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.418+0000 7fa764a08640 1 -- 172.21.15.163:0/3581169072 shutdown_connections 2024-05-08T06:38:11.419 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.418+0000 7fa764a08640 1 -- 172.21.15.163:0/3581169072 wait complete. 2024-05-08T06:38:11.428 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-05-08T06:38:10.115022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220480,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754240,"statfs":{"total":773094113280,"available":772868341760,"internally_reserved":0,"allocated":4431872,"data_stored":2445648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002696"},"pg_stats":[{"pgid":"1.0","version":"35'74","reported_seq":149,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-08T06:37:51.696006+0000","last_change":"2024-05-08T06:37:51.696042+0000","last_active":"2024-05-08T06:37:51.696006+0000","last_peered":"2024-05-08T06:37:51.696006+0000","last_clean":"2024-05-08T06:37:51.696006+0000","last_became_active":"2024-05-08T06:36:31.709270+0000","last_became_peered":"2024-05-08T06:36:31.709270+0000","last_unstale":"2024-05-08T06:37:51.696006+0000","last_undegraded":"2024-05-08T06:37:51.696006+0000","last_fullsized":"2024-05-08T06:37:51.696006+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'74","last_scrub_stamp":"2024-05-08T06:37:51.695920+0000","last_deep_scrub":"35'74","last_deep_scrub_stamp":"2024-05-08T06:37:51.695920+0000","last_clean_scrub_stamp":"2024-05-08T06:37:51.695920+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:49.771117+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":999424,"data_stored":748482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":999424,"data_stored":748482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:38:11.429 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-08T06:38:11.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.529+0000 7f6360914640 1 Processor -- start 2024-05-08T06:38:11.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.529+0000 7f6360914640 1 -- start start 2024-05-08T06:38:11.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.530+0000 7f6360914640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f635c0d3720 0x7f635c15d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.530+0000 7f6360914640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f635c15deb0 0x7f635c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.530+0000 7f6360914640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f635c15e6d0 con 0x7f635c140050 2024-05-08T06:38:11.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.530+0000 7f6360914640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f635c071e30 con 0x7f635c15deb0 2024-05-08T06:38:11.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.530+0000 7f6360914640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f635c072110 con 0x7f635c0d3720 2024-05-08T06:38:11.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.530+0000 7f635ad9d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f635c15deb0 0x7f635c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.530+0000 7f635a59c640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f635c140050 0x7f635c0d3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:55510/0 (socket says 172.21.15.163:55510) 2024-05-08T06:38:11.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.530+0000 7f635a59c640 1 -- 172.21.15.163:0/4272938557 learned_addr learned my addr 172.21.15.163:0/4272938557 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:38:11.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.531+0000 7f635ad9d640 1 -- 172.21.15.163:0/4272938557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f635c0d3720 msgr2=0x7f635c15d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:38:11.532 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.531+0000 7f635ad9d640 1 --2- 172.21.15.163:0/4272938557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f635c0d3720 0x7f635c15d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.532 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.531+0000 7f6359d9b640 1 -- 172.21.15.163:0/4272938557 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2595802788 0 0) 0x7f635c15e6d0 con 0x7f635c140050 2024-05-08T06:38:11.532 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.531+0000 7f635ad9d640 1 -- 172.21.15.163:0/4272938557 >> v1:172.21.15.163:6789/0 conn(0x7f635c140050 legacy=0x7f635c0d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.532 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.531+0000 7f635ad9d640 1 -- 172.21.15.163:0/4272938557 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635c0723f0 con 0x7f635c15deb0 2024-05-08T06:38:11.532 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.531+0000 7f635ad9d640 1 --2- 172.21.15.163:0/4272938557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c15deb0 0x7f635c0d19f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6354009d90 tx=0x7f635402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b09a825c1d770f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:38:11.532 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.532+0000 7f6359d9b640 1 -- 172.21.15.163:0/4272938557 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354007740 con 0x7f635c15deb0 2024-05-08T06:38:11.532 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.532+0000 7f6359d9b640 1 -- 172.21.15.163:0/4272938557 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63540078e0 con 0x7f635c15deb0 2024-05-08T06:38:11.532 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.532+0000 7f6359d9b640 1 -- 172.21.15.163:0/4272938557 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354007cb0 con 0x7f635c15deb0 2024-05-08T06:38:11.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.532+0000 7f6360914640 1 -- 172.21.15.163:0/4272938557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c15deb0 msgr2=0x7f635c0d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.532+0000 7f6360914640 1 --2- 172.21.15.163:0/4272938557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c15deb0 0x7f635c0d19f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6354009d90 tx=0x7f635402a660 comp rx=0 tx=0).stop 2024-05-08T06:38:11.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.532+0000 7f6360914640 1 -- 172.21.15.163:0/4272938557 shutdown_connections 2024-05-08T06:38:11.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.532+0000 7f6360914640 1 --2- 172.21.15.163:0/4272938557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c15deb0 0x7f635c0d19f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.532+0000 7f6360914640 1 --2- 172.21.15.163:0/4272938557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f635c0d3720 0x7f635c15d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.532+0000 7f6360914640 1 -- 172.21.15.163:0/4272938557 >> 172.21.15.163:0/4272938557 conn(0x7f635c0b4d30 msgr2=0x7f635c0b5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:38:11.535 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.532+0000 7f6360914640 1 -- 172.21.15.163:0/4272938557 shutdown_connections 2024-05-08T06:38:11.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.532+0000 7f6360914640 1 -- 172.21.15.163:0/4272938557 wait complete. 2024-05-08T06:38:11.536 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.534+0000 7f6360914640 1 Processor -- start 2024-05-08T06:38:11.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.534+0000 7f6360914640 1 -- start start 2024-05-08T06:38:11.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.534+0000 7f6360914640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f635c0d3720 0x7f635c15c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.534+0000 7f6360914640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c140050 0x7f635c155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.534+0000 7f6360914640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f635c15deb0 0x7f635c15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.534+0000 7f6360914640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f635c072bf0 con 0x7f635c140050 2024-05-08T06:38:11.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.534+0000 7f6360914640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f635c15e320 con 0x7f635c0d3720 2024-05-08T06:38:11.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.534+0000 7f6360914640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f635c15b2e0 con 0x7f635c15deb0 2024-05-08T06:38:11.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.535+0000 7f6351d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c140050 0x7f635c155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.535+0000 7f6351d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c140050 0x7f635c155a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40698/0 (socket says 172.21.15.163:40698) 2024-05-08T06:38:11.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.535+0000 7f6351d9b640 1 -- 172.21.15.163:0/1653362283 learned_addr learned my addr 172.21.15.163:0/1653362283 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:38:11.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.535+0000 7f6351d9b640 1 -- 172.21.15.163:0/1653362283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f635c15deb0 msgr2=0x7f635c15a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.535+0000 7f6351d9b640 1 --2- 172.21.15.163:0/1653362283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f635c15deb0 0x7f635c15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.535+0000 7f6351d9b640 1 -- 172.21.15.163:0/1653362283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f635c0d3720 msgr2=0x7f635c15c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.538 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.535+0000 7f6351d9b640 1 --2- 172.21.15.163:0/1653362283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f635c0d3720 0x7f635c15c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.535+0000 7f6351d9b640 1 -- 172.21.15.163:0/1653362283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635c15b530 con 0x7f635c140050 2024-05-08T06:38:11.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.535+0000 7f6351d9b640 1 --2- 172.21.15.163:0/1653362283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c140050 0x7f635c155a20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f634400b4a0 tx=0x7f634400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:38:11.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.535+0000 7f63537fe640 1 -- 172.21.15.163:0/1653362283 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344014070 con 0x7f635c140050 2024-05-08T06:38:11.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.535+0000 7f63537fe640 1 -- 172.21.15.163:0/1653362283 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6344010610 con 0x7f635c140050 2024-05-08T06:38:11.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.535+0000 7f6360914640 1 -- 172.21.15.163:0/1653362283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635c15b810 con 0x7f635c140050 2024-05-08T06:38:11.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.535+0000 7f63537fe640 1 -- 172.21.15.163:0/1653362283 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344010a00 con 0x7f635c140050 2024-05-08T06:38:11.539 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.535+0000 7f6360914640 1 -- 172.21.15.163:0/1653362283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f635c15bdd0 con 0x7f635c140050 2024-05-08T06:38:11.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.536+0000 7f6360914640 1 -- 172.21.15.163:0/1653362283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f635c06a550 con 0x7f635c140050 2024-05-08T06:38:11.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.537+0000 7f63537fe640 1 -- 172.21.15.163:0/1653362283 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6344017070 con 0x7f635c140050 2024-05-08T06:38:11.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.537+0000 7f63537fe640 1 --2- 172.21.15.163:0/1653362283 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f633c0762d0 0x7f633c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.538+0000 7f635a59c640 1 --2- 172.21.15.163:0/1653362283 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f633c0762d0 0x7f633c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.538+0000 7f63537fe640 1 -- 172.21.15.163:0/1653362283 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f6344096d40 con 0x7f635c140050 2024-05-08T06:38:11.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.538+0000 7f635a59c640 1 --2- 172.21.15.163:0/1653362283 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f633c0762d0 0x7f633c078790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f635c0d0220 tx=0x7f634c0078d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:38:11.541 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.542+0000 7f63537fe640 1 -- 172.21.15.163:0/1653362283 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6344060950 con 0x7f635c140050 2024-05-08T06:38:11.723 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.723+0000 7f6360914640 1 -- 172.21.15.163:0/1653362283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f635c0d3d60 con 0x7f635c140050 2024-05-08T06:38:11.725 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.727+0000 7f63537fe640 1 -- 172.21.15.163:0/1653362283 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f6344060250 con 0x7f635c140050 2024-05-08T06:38:11.726 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.727+0000 7f63537fe640 1 -- 172.21.15.163:0/1653362283 <== mon.0 v2:172.21.15.163:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f634405fd60 con 0x7f635c140050 2024-05-08T06:38:11.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.729+0000 7f6360914640 1 -- 172.21.15.163:0/1653362283 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f633c0762d0 msgr2=0x7f633c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.729+0000 7f6360914640 1 --2- 172.21.15.163:0/1653362283 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f633c0762d0 0x7f633c078790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f635c0d0220 tx=0x7f634c0078d0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.729+0000 7f6360914640 1 -- 172.21.15.163:0/1653362283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c140050 msgr2=0x7f635c155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.729+0000 7f6360914640 1 --2- 172.21.15.163:0/1653362283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c140050 0x7f635c155a20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f634400b4a0 tx=0x7f634400b960 comp rx=0 tx=0).stop 2024-05-08T06:38:11.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.729+0000 7f6360914640 1 -- 172.21.15.163:0/1653362283 shutdown_connections 2024-05-08T06:38:11.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.729+0000 7f6360914640 1 --2- 172.21.15.163:0/1653362283 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7f633c0762d0 0x7f633c078790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.729 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.729+0000 7f6360914640 1 --2- 172.21.15.163:0/1653362283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f635c15deb0 0x7f635c15a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.729+0000 7f6360914640 1 --2- 172.21.15.163:0/1653362283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f635c140050 0x7f635c155a20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.729+0000 7f6360914640 1 --2- 172.21.15.163:0/1653362283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f635c0d3720 0x7f635c15c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.729+0000 7f6360914640 1 -- 172.21.15.163:0/1653362283 >> 172.21.15.163:0/1653362283 conn(0x7f635c0b4d30 msgr2=0x7f635c0b7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:38:11.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.730+0000 7f6360914640 1 -- 172.21.15.163:0/1653362283 shutdown_connections 2024-05-08T06:38:11.730 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.730+0000 7f6360914640 1 -- 172.21.15.163:0/1653362283 wait complete. 2024-05-08T06:38:11.740 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T06:38:11.740 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T06:38:11.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.841+0000 7fed3e346640 1 Processor -- start 2024-05-08T06:38:11.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.841+0000 7fed3e346640 1 -- start start 2024-05-08T06:38:11.844 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.842+0000 7fed3e346640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fed381493c0 0x7fed380afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.844 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.842+0000 7fed3e346640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed380b0560 con 0x7fed380705f0 2024-05-08T06:38:11.844 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.842+0000 7fed3e346640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed380b0840 con 0x7fed3806da40 2024-05-08T06:38:11.844 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.842+0000 7fed3e346640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fed3815fde0 con 0x7fed381493c0 2024-05-08T06:38:11.844 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.842+0000 7fed377fe640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fed380705f0 0x7fed3806d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:55524/0 (socket says 172.21.15.163:55524) 2024-05-08T06:38:11.844 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.842+0000 7fed377fe640 1 -- 172.21.15.163:0/108264806 learned_addr learned my addr 172.21.15.163:0/108264806 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-08T06:38:11.845 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.842+0000 7fed37fff640 1 --2- 172.21.15.163:0/108264806 >> v2:172.21.15.192:3300/0 conn(0x7fed381493c0 0x7fed380afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.845 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.842+0000 7fed36ffd640 1 -- 172.21.15.163:0/108264806 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3824971201 0 0) 0x7fed380b0560 con 0x7fed380705f0 2024-05-08T06:38:11.845 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.842+0000 7fed36ffd640 1 -- 172.21.15.163:0/108264806 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed20003520 con 0x7fed380705f0 2024-05-08T06:38:11.845 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed36ffd640 1 -- 172.21.15.163:0/108264806 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1751146714 0 0) 0x7fed20003520 con 0x7fed380705f0 2024-05-08T06:38:11.845 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed36ffd640 1 -- 172.21.15.163:0/108264806 >> v2:172.21.15.192:3300/0 conn(0x7fed381493c0 msgr2=0x7fed380afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:38:11.845 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed36ffd640 1 --2- 172.21.15.163:0/108264806 >> v2:172.21.15.192:3300/0 conn(0x7fed381493c0 0x7fed380afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.845 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed36ffd640 1 -- 172.21.15.163:0/108264806 >> v1:172.21.15.192:6789/0 conn(0x7fed3806da40 legacy=0x7fed380b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.845 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed36ffd640 1 -- 172.21.15.163:0/108264806 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed380b0b20 con 0x7fed380705f0 2024-05-08T06:38:11.845 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed36ffd640 1 -- 172.21.15.163:0/108264806 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1471908829 0 0) 0x7fed2c0016f0 con 0x7fed380705f0 2024-05-08T06:38:11.846 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed36ffd640 1 -- 172.21.15.163:0/108264806 >> v1:172.21.15.163:6789/0 conn(0x7fed380705f0 legacy=0x7fed3806d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.846 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed36ffd640 1 --2- 172.21.15.163:0/108264806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed20003a50 0x7fed20003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.846 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed36ffd640 1 --2- 172.21.15.163:0/108264806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed20004800 0x7fed20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.846 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed36ffd640 1 --2- 172.21.15.163:0/108264806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed200052e0 0x7fed2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.846 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed36ffd640 1 -- 172.21.15.163:0/108264806 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fed3815fde0 con 0x7fed200052e0 2024-05-08T06:38:11.846 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed36ffd640 1 -- 172.21.15.163:0/108264806 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fed2000c1b0 con 0x7fed20004800 2024-05-08T06:38:11.846 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed36ffd640 1 -- 172.21.15.163:0/108264806 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fed2000c3b0 con 0x7fed20003a50 2024-05-08T06:38:11.846 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed3c8e3640 1 --2- 172.21.15.163:0/108264806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed200052e0 0x7fed2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.847 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed37fff640 1 --2- 172.21.15.163:0/108264806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed20003a50 0x7fed20003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.847 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed377fe640 1 --2- 172.21.15.163:0/108264806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed20004800 0x7fed20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.847 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed3c8e3640 1 -- 172.21.15.163:0/108264806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed20003a50 msgr2=0x7fed20003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:38:11.847 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed3c8e3640 1 --2- 172.21.15.163:0/108264806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed20003a50 0x7fed20003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.847 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed3c8e3640 1 -- 172.21.15.163:0/108264806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed20004800 msgr2=0x7fed20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:38:11.847 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed3c8e3640 1 --2- 172.21.15.163:0/108264806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed20004800 0x7fed20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.847 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.843+0000 7fed3c8e3640 1 -- 172.21.15.163:0/108264806 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed20003520 con 0x7fed200052e0 2024-05-08T06:38:11.847 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.844+0000 7fed3c8e3640 1 -- 172.21.15.163:0/108264806 reap_dead start 2024-05-08T06:38:11.847 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.844+0000 7fed3c8e3640 1 --2- 172.21.15.163:0/108264806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed200052e0 0x7fed2000b9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fed30001e90 tx=0x7fed20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd8231f1a756100 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:38:11.848 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.844+0000 7fed36ffd640 1 -- 172.21.15.163:0/108264806 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30001ab0 con 0x7fed200052e0 2024-05-08T06:38:11.848 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.844+0000 7fed36ffd640 1 -- 172.21.15.163:0/108264806 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fed3002ae50 con 0x7fed200052e0 2024-05-08T06:38:11.848 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.844+0000 7fed36ffd640 1 -- 172.21.15.163:0/108264806 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30003bd0 con 0x7fed200052e0 2024-05-08T06:38:11.848 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.844+0000 7fed3e346640 1 -- 172.21.15.163:0/108264806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed200052e0 msgr2=0x7fed2000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.848 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.844+0000 7fed3e346640 1 --2- 172.21.15.163:0/108264806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed200052e0 0x7fed2000b9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fed30001e90 tx=0x7fed20005830 comp rx=0 tx=0).stop 2024-05-08T06:38:11.848 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.844+0000 7fed3e346640 1 -- 172.21.15.163:0/108264806 shutdown_connections 2024-05-08T06:38:11.848 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.844+0000 7fed3e346640 1 --2- 172.21.15.163:0/108264806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed200052e0 0x7fed2000b9c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.848 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.844+0000 7fed3e346640 1 -- 172.21.15.163:0/108264806 >> 172.21.15.163:0/108264806 conn(0x7fed38072ba0 msgr2=0x7fed38072f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:38:11.849 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.845+0000 7fed3e346640 1 -- 172.21.15.163:0/108264806 shutdown_connections 2024-05-08T06:38:11.849 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.845+0000 7fed3e346640 1 -- 172.21.15.163:0/108264806 wait complete. 2024-05-08T06:38:11.851 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.846+0000 7fed3e346640 1 Processor -- start 2024-05-08T06:38:11.851 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.846+0000 7fed3e346640 1 -- start start 2024-05-08T06:38:11.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3e346640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed200052e0 0x7fed381f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3e346640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed381483f0 0x7fed38162430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3e346640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed38148870 0x7fed38155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3e346640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fed3806d4b0 con 0x7fed38148870 2024-05-08T06:38:11.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3e346640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fed3806d330 con 0x7fed381483f0 2024-05-08T06:38:11.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3e346640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fed3806d630 con 0x7fed200052e0 2024-05-08T06:38:11.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3c8e3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed38148870 0x7fed38155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3c8e3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed38148870 0x7fed38155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40720/0 (socket says 172.21.15.163:40720) 2024-05-08T06:38:11.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3c8e3640 1 -- 172.21.15.163:0/3907485803 learned_addr learned my addr 172.21.15.163:0/3907485803 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:38:11.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed377fe640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed381483f0 0x7fed38162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed37fff640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed200052e0 0x7fed381f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3c8e3640 1 -- 172.21.15.163:0/3907485803 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed200052e0 msgr2=0x7fed381f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3c8e3640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed200052e0 0x7fed381f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3c8e3640 1 -- 172.21.15.163:0/3907485803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed381483f0 msgr2=0x7fed38162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:11.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3c8e3640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed381483f0 0x7fed38162430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:11.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3c8e3640 1 -- 172.21.15.163:0/3907485803 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed381f5950 con 0x7fed38148870 2024-05-08T06:38:11.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed37fff640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed200052e0 0x7fed381f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:38:11.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3c8e3640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed38148870 0x7fed38155b90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fed3002a900 tx=0x7fed300014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:38:11.854 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed34ff9640 1 -- 172.21.15.163:0/3907485803 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30007df0 con 0x7fed38148870 2024-05-08T06:38:11.854 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed34ff9640 1 -- 172.21.15.163:0/3907485803 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fed3002ac50 con 0x7fed38148870 2024-05-08T06:38:11.854 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3e346640 1 -- 172.21.15.163:0/3907485803 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3813e910 con 0x7fed38148870 2024-05-08T06:38:11.854 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed34ff9640 1 -- 172.21.15.163:0/3907485803 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30005540 con 0x7fed38148870 2024-05-08T06:38:11.854 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.847+0000 7fed3e346640 1 -- 172.21.15.163:0/3907485803 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3813eed0 con 0x7fed38148870 2024-05-08T06:38:11.854 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.848+0000 7fed3e346640 1 -- 172.21.15.163:0/3907485803 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed3806a550 con 0x7fed38148870 2024-05-08T06:38:11.854 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.849+0000 7fed34ff9640 1 -- 172.21.15.163:0/3907485803 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fed30044020 con 0x7fed38148870 2024-05-08T06:38:11.854 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.849+0000 7fed34ff9640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fed240762d0 0x7fed24078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:38:11.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.849+0000 7fed37fff640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fed240762d0 0x7fed24078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:38:11.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.849+0000 7fed34ff9640 1 -- 172.21.15.163:0/3907485803 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fed3002ae00 con 0x7fed38148870 2024-05-08T06:38:11.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.850+0000 7fed37fff640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fed240762d0 0x7fed24078790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fed28009b40 tx=0x7fed28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:38:11.855 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:11.854+0000 7fed34ff9640 1 -- 172.21.15.163:0/3907485803 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fed30080040 con 0x7fed38148870 2024-05-08T06:38:12.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:12.071+0000 7fed3e346640 1 -- 172.21.15.163:0/3907485803 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed38073a90 con 0x7fed38148870 2024-05-08T06:38:12.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:12.071+0000 7fed34ff9640 1 -- 172.21.15.163:0/3907485803 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+3881 (secure 0 0 0) 0x7fed3007f940 con 0x7fed38148870 2024-05-08T06:38:12.071 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T06:38:12.071 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1773142301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1773142301},{"type":"v1","addr":"172.21.15.163:6835","nonce":1773142301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2952930735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2952930735},{"type":"v1","addr":"172.21.15.192:6834","nonce":2952930735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4583,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1297249355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1297249355},{"type":"v1","addr":"172.21.15.192:6835","nonce":1297249355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5148,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2452503459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":2452503459},{"type":"v1","addr":"172.21.15.163:6837","nonce":2452503459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[]} 2024-05-08T06:38:12.072 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 22 2024-05-08T06:38:12.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:12.073+0000 7fed3e346640 1 -- 172.21.15.163:0/3907485803 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fed240762d0 msgr2=0x7fed24078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:12.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:12.073+0000 7fed3e346640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fed240762d0 0x7fed24078790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fed28009b40 tx=0x7fed28008040 comp rx=0 tx=0).stop 2024-05-08T06:38:12.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:12.073+0000 7fed3e346640 1 -- 172.21.15.163:0/3907485803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed38148870 msgr2=0x7fed38155b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:38:12.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:12.073+0000 7fed3e346640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed38148870 0x7fed38155b90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fed3002a900 tx=0x7fed300014a0 comp rx=0 tx=0).stop 2024-05-08T06:38:12.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:12.073+0000 7fed3e346640 1 -- 172.21.15.163:0/3907485803 shutdown_connections 2024-05-08T06:38:12.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:12.074+0000 7fed3e346640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.163:6832/2781435851,v1:172.21.15.163:6833/2781435851] conn(0x7fed240762d0 0x7fed24078790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:12.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:12.074+0000 7fed3e346640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed38148870 0x7fed38155b90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:12.074 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:12.074+0000 7fed3e346640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed381483f0 0x7fed38162430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:12.074 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:12.074+0000 7fed3e346640 1 --2- 172.21.15.163:0/3907485803 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed200052e0 0x7fed381f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:38:12.074 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:12.074+0000 7fed3e346640 1 -- 172.21.15.163:0/3907485803 >> 172.21.15.163:0/3907485803 conn(0x7fed38072ba0 msgr2=0x7fed38146700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:38:12.074 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:12.074+0000 7fed3e346640 1 -- 172.21.15.163:0/3907485803 shutdown_connections 2024-05-08T06:38:12.074 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:38:12.074+0000 7fed3e346640 1 -- 172.21.15.163:0/3907485803 wait complete. 2024-05-08T06:38:12.084 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-08T06:38:12.084 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-08T06:38:12.084 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T06:38:12.084 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.093 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T06:38:12.094 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T06:38:12.094 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.138 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T06:38:12.140 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T06:38:12.143 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T06:38:12.143 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T06:38:12.143 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.155 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T06:38:12.155 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T06:38:12.155 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.205 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T06:38:12.206 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-08T06:38:12.206 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-08T06:38:12.206 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.241 INFO:tasks.ceph.osd.0:Stopped 2024-05-08T06:38:12.241 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T06:38:12.241 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.306 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T06:38:12.307 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-08T06:38:12.307 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.385 INFO:tasks.ceph.osd.2:Stopped 2024-05-08T06:38:12.385 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-08T06:38:12.385 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.457 INFO:tasks.ceph.osd.3:Stopped 2024-05-08T06:38:12.458 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-08T06:38:12.458 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.492 INFO:tasks.ceph.osd.4:Stopped 2024-05-08T06:38:12.492 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T06:38:12.492 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.569 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T06:38:12.569 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-08T06:38:12.569 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.635 INFO:tasks.ceph.osd.6:Stopped 2024-05-08T06:38:12.635 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-08T06:38:12.635 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.707 INFO:tasks.ceph.osd.7:Stopped 2024-05-08T06:38:12.707 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-08T06:38:12.707 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-08T06:38:12.707 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.733 INFO:tasks.ceph.mgr.y:Stopped 2024-05-08T06:38:12.733 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-08T06:38:12.733 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.774 INFO:tasks.ceph.mgr.x:Stopped 2024-05-08T06:38:12.774 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-08T06:38:12.774 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T06:38:12.775 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.785 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T06:38:12.785 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T06:38:12.785 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.825 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T06:38:12.826 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T06:38:12.826 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T06:38:12.877 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T06:38:12.877 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-08T06:38:12.878 DEBUG:teuthology.orchestra.run.smithi163:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-08T06:38:12.918 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:38:12.918 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-08T06:38:13.130 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:38:13.131 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-08T06:38:13.235 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:38:13.236 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-08T06:38:13.293 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:38:13.293 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-08T06:38:13.360 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T06:38:13.360 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-08T06:38:13.520 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T06:38:13.521 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-08T06:38:13.625 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T06:38:13.626 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-08T06:38:13.692 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T06:38:13.692 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-08T06:38:13.749 INFO:tasks.ceph:Archiving mon data... 2024-05-08T06:38:13.775 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389/data/mon.a.tgz 2024-05-08T06:38:13.776 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2024-05-08T06:38:13.796 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.myTqhUD1pH 2024-05-08T06:38:13.797 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.myTqhUD1pH 2024-05-08T06:38:13.931 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.myTqhUD1pH 2024-05-08T06:38:14.016 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.myTqhUD1pH is 284KB 2024-05-08T06:38:14.138 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.myTqhUD1pH 2024-05-08T06:38:14.154 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389/data/mon.b.tgz 2024-05-08T06:38:14.154 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-05-08T06:38:14.174 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.4fYvPsMwyr 2024-05-08T06:38:14.174 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.4fYvPsMwyr 2024-05-08T06:38:14.324 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.4fYvPsMwyr 2024-05-08T06:38:14.409 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.4fYvPsMwyr is 385KB 2024-05-08T06:38:14.478 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.4fYvPsMwyr 2024-05-08T06:38:14.493 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389/data/mon.c.tgz 2024-05-08T06:38:14.493 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-05-08T06:38:14.548 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.aj44N0XeKa 2024-05-08T06:38:14.549 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.aj44N0XeKa 2024-05-08T06:38:14.682 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.aj44N0XeKa 2024-05-08T06:38:14.765 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.aj44N0XeKa is 385KB 2024-05-08T06:38:14.861 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.aj44N0XeKa 2024-05-08T06:38:14.876 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-08T06:38:14.876 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T06:38:14.879 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T06:38:14.960 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T06:38:14.996 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T06:38:14.996 INFO:tasks.ceph:Archiving crash dumps... 2024-05-08T06:38:14.998 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/crash to /home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389/remote/smithi163/crash 2024-05-08T06:38:14.999 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T06:38:15.027 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/crash to /home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389/remote/smithi192/crash 2024-05-08T06:38:15.029 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T06:38:15.060 INFO:tasks.ceph:Shutting down logrotate 2024-05-08T06:38:15.061 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T06:38:15.073 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T06:38:15.154 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-08T06:38:15.181 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-08T06:38:15.206 INFO:tasks.ceph:Compressing logs... 2024-05-08T06:38:15.206 DEBUG:teuthology.orchestra.run.smithi163:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T06:38:15.224 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T06:38:15.247 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44936.log 2024-05-08T06:38:15.247 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43797.log 2024-05-08T06:38:15.291 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35594.log 2024-05-08T06:38:15.291 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41905.log 2024-05-08T06:38:15.291 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35422.log 2024-05-08T06:38:15.292 INFO:teuthology.orchestra.run.smithi192.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35594.log.gz 2024-05-08T06:38:15.292 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.1.41905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35674.log 0.0% 2024-05-08T06:38:15.292 INFO:teuthology.orchestra.run.smithi192.stderr: -- replaced with /var/log/ceph/ceph-client.1.41905.log.gz 2024-05-08T06:38:15.292 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41910.log 2024-05-08T06:38:15.292 INFO:teuthology.orchestra.run.smithi192.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35422.log.gz 2024-05-08T06:38:15.292 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35674.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T06:38:15.292 INFO:teuthology.orchestra.run.smithi192.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35674.log.gz 2024-05-08T06:38:15.292 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.1.41910.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T06:38:15.292 INFO:teuthology.orchestra.run.smithi192.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41910.log.gz 2024-05-08T06:38:15.293 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-08T06:38:15.293 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35554.log 2024-05-08T06:38:15.293 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-08T06:38:15.293 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35554.log: 91.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T06:38:15.293 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-08T06:38:15.293 INFO:teuthology.orchestra.run.smithi192.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35554.log.gz 2024-05-08T06:38:15.293 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35714.log 2024-05-08T06:38:15.293 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-08T06:38:15.294 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35714.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T06:38:15.294 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35634.log 2024-05-08T06:38:15.294 INFO:teuthology.orchestra.run.smithi192.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-08T06:38:15.294 INFO:teuthology.orchestra.run.smithi192.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35714.log.gz 2024-05-08T06:38:15.294 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42310.log 2024-05-08T06:38:15.294 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35634.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-08T06:38:15.294 INFO:teuthology.orchestra.run.smithi192.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35634.log.gz 2024-05-08T06:38:15.294 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.3.42310.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42310.log.gz 2024-05-08T06:38:15.295 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-08T06:38:15.295 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-08T06:38:15.295 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35502.log 2024-05-08T06:38:15.295 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35462.log 2024-05-08T06:38:15.295 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42306.log 2024-05-08T06:38:15.295 INFO:teuthology.orchestra.run.smithi192.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35502.log.gz 2024-05-08T06:38:15.295 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35462.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35462.log.gz 2024-05-08T06:38:15.295 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-08T06:38:15.296 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.3.42306.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42306.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-08T06:38:15.296 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-05-08T06:38:15.296 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 92.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-08T06:38:15.329 INFO:teuthology.orchestra.run.smithi192.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-08T06:38:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35576.log 2024-05-08T06:38:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.44936.log: /var/log/ceph/ceph-client.admin.43797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45544.log 2024-05-08T06:38:15.334 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44936.log.gz 2024-05-08T06:38:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.35576.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43797.log.gz 2024-05-08T06:38:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42033.log 2024-05-08T06:38:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.45544.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35576.log.gz 2024-05-08T06:38:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47891.log 2024-05-08T06:38:15.335 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8%/var/log/ceph/ceph-client.admin.42033.log: -- replaced with /var/log/ceph/ceph-client.admin.45544.log.gz 2024-05-08T06:38:15.335 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43001.log 2024-05-08T06:38:15.335 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.47891.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42033.log.gz 2024-05-08T06:38:15.335 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43587.log 2024-05-08T06:38:15.335 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42327.log 2024-05-08T06:38:15.335 INFO:teuthology.orchestra.run.smithi163.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47891.log.gz 2024-05-08T06:38:15.335 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47623.log 2024-05-08T06:38:15.335 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43001.log.gz 2024-05-08T06:38:15.335 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43587.log.gz/var/log/ceph/ceph-client.admin.42327.log: 2024-05-08T06:38:15.336 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43209.log 2024-05-08T06:38:15.336 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.47623.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42327.log.gz 2024-05-08T06:38:15.336 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49414.log 2024-05-08T06:38:15.336 INFO:teuthology.orchestra.run.smithi163.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.47623.log.gz 2024-05-08T06:38:15.336 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44453.log 2024-05-08T06:38:15.336 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43209.log.gz 2024-05-08T06:38:15.336 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45898.log 2024-05-08T06:38:15.336 INFO:teuthology.orchestra.run.smithi163.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49414.log.gz/var/log/ceph/ceph-client.admin.44453.log: 2024-05-08T06:38:15.336 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49918.log 2024-05-08T06:38:15.336 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.45898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45754.log 2024-05-08T06:38:15.337 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44453.log.gz 2024-05-08T06:38:15.337 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49582.log 2024-05-08T06:38:15.337 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45898.log.gz 2024-05-08T06:38:15.337 INFO:teuthology.orchestra.run.smithi163.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49918.log.gz 2024-05-08T06:38:15.337 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.45754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43977.log 2024-05-08T06:38:15.337 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49582.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45754.log.gz 2024-05-08T06:38:15.337 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50000.log 2024-05-08T06:38:15.337 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49582.log.gz 2024-05-08T06:38:15.337 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46155.log 2024-05-08T06:38:15.338 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.50000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48154.log 2024-05-08T06:38:15.338 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43977.log.gz 2024-05-08T06:38:15.338 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50000.log.gz 2024-05-08T06:38:15.338 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41593.log 2024-05-08T06:38:15.338 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5%/var/log/ceph/ceph-client.admin.48154.log: -- replaced with /var/log/ceph/ceph-client.admin.46155.log.gz 2024-05-08T06:38:15.338 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49666.log 2024-05-08T06:38:15.338 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.41593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47759.log 2024-05-08T06:38:15.338 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41593.log.gz 2024-05-08T06:38:15.339 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49666.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48154.log.gz 2024-05-08T06:38:15.339 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43348.log 2024-05-08T06:38:15.339 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.47759.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49666.log.gz 2024-05-08T06:38:15.339 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49330.log 2024-05-08T06:38:15.339 INFO:teuthology.orchestra.run.smithi163.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.47759.log.gz 2024-05-08T06:38:15.339 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49372.log 2024-05-08T06:38:15.339 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T06:38:15.339 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49330.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43348.log.gz 2024-05-08T06:38:15.339 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45670.log 2024-05-08T06:38:15.340 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49330.log.gz 2024-05-08T06:38:15.340 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49372.log.gz 2024-05-08T06:38:15.340 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48689.log 2024-05-08T06:38:15.340 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.45670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43671.log 2024-05-08T06:38:15.340 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.48689.log: 85.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T06:38:15.340 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45670.log.gz 2024-05-08T06:38:15.340 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42201.log 2024-05-08T06:38:15.340 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43671.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48689.log.gz 2024-05-08T06:38:15.340 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41889.log 2024-05-08T06:38:15.341 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43671.log.gz 2024-05-08T06:38:15.341 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.42201.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-08T06:38:15.341 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.41889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45814.log 2024-05-08T06:38:15.341 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42201.log.gz 2024-05-08T06:38:15.341 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.2.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41889.log.gz 2024-05-08T06:38:15.341 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44978.log 2024-05-08T06:38:15.341 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.45814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47055.log 2024-05-08T06:38:15.341 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.44978.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45814.log.gz 2024-05-08T06:38:15.341 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49456.log 2024-05-08T06:38:15.341 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44978.log.gz 2024-05-08T06:38:15.342 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.47055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50042.log 2024-05-08T06:38:15.342 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49834.log 2024-05-08T06:38:15.342 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49456.log.gz 2024-05-08T06:38:15.342 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.50042.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47055.log.gz 2024-05-08T06:38:15.342 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49245.log 2024-05-08T06:38:15.342 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43012.log 2024-05-08T06:38:15.342 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49834.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50042.log.gz 2024-05-08T06:38:15.342 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5/var/log/ceph/ceph-client.admin.49245.log: --verbose -- /var/log/ceph/ceph-client.2.44757.log 2024-05-08T06:38:15.343 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49834.log.gz 2024-05-08T06:38:15.343 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43012.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49245.log.gz 2024-05-08T06:38:15.343 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T06:38:15.343 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43012.log.gz 2024-05-08T06:38:15.343 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.2.44757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43013.log 2024-05-08T06:38:15.343 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44757.log.gz 2024-05-08T06:38:15.343 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42411.log 2024-05-08T06:38:15.343 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49192.log 2024-05-08T06:38:15.343 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.42411.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43013.log.gz 2024-05-08T06:38:15.343 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47722.log 2024-05-08T06:38:15.344 INFO:teuthology.orchestra.run.smithi163.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.42411.log.gz 2024-05-08T06:38:15.344 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49150.log 2024-05-08T06:38:15.344 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49192.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T06:38:15.344 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.47722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48545.log 2024-05-08T06:38:15.344 INFO:teuthology.orchestra.run.smithi163.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49192.log.gz 2024-05-08T06:38:15.344 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49150.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.47722.log.gz 2024-05-08T06:38:15.344 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46971.log 2024-05-08T06:38:15.344 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49150.log.gz 2024-05-08T06:38:15.344 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.48545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35679.log 2024-05-08T06:38:15.345 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48545.log.gz 2024-05-08T06:38:15.345 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49708.log 2024-05-08T06:38:15.345 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46971.log: /var/log/ceph/ceph-client.admin.35679.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-08T06:38:15.345 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46971.log.gz 2024-05-08T06:38:15.345 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35679.log.gz 2024-05-08T06:38:15.345 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43007.log 2024-05-08T06:38:15.345 INFO:teuthology.orchestra.run.smithi163.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49708.log.gz 2024-05-08T06:38:15.345 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46073.log 2024-05-08T06:38:15.345 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46845.log 2024-05-08T06:38:15.346 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46073.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43007.log.gz 2024-05-08T06:38:15.346 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46366.log 2024-05-08T06:38:15.346 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46845.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46073.log.gz 2024-05-08T06:38:15.346 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42675.log 2024-05-08T06:38:15.346 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46845.log.gz 2024-05-08T06:38:15.346 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47970.log 2024-05-08T06:38:15.346 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46366.log: /var/log/ceph/ceph-client.admin.42675.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-08T06:38:15.347 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46366.log.gz 2024-05-08T06:38:15.347 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.47970.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42675.log.gz 2024-05-08T06:38:15.347 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45856.log 2024-05-08T06:38:15.347 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46580.log 2024-05-08T06:38:15.347 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mds.c.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47970.log.gz 2024-05-08T06:38:15.347 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.45856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43005.log 2024-05-08T06:38:15.347 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45856.log.gz 2024-05-08T06:38:15.347 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46580.log: 91.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gzgzip 2024-05-08T06:38:15.347 INFO:teuthology.orchestra.run.smithi163.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-08T06:38:15.347 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43005.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-08T06:38:15.348 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46580.log.gz 2024-05-08T06:38:15.348 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49287.log 2024-05-08T06:38:15.348 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43005.log.gz 2024-05-08T06:38:15.348 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41759.log 2024-05-08T06:38:15.348 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42674.log 2024-05-08T06:38:15.348 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49287.log.gz 2024-05-08T06:38:15.348 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.41759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48024.log 2024-05-08T06:38:15.348 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.42674.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41759.log.gz 2024-05-08T06:38:15.349 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45982.log 2024-05-08T06:38:15.349 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42674.log.gz 2024-05-08T06:38:15.349 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.48024.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-08T06:38:15.349 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48024.log.gz 2024-05-08T06:38:15.349 INFO:teuthology.orchestra.run.smithi163.stderr:gzip/var/log/ceph/ceph-client.admin.45982.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46282.log 2024-05-08T06:38:15.349 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.0.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46408.log 2024-05-08T06:38:15.349 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45982.log.gz 2024-05-08T06:38:15.349 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48220.log 2024-05-08T06:38:15.349 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46282.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46715.log 2024-05-08T06:38:15.350 INFO:teuthology.orchestra.run.smithi163.stderr: 2024-05-08T06:38:15.350 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48263.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46408.log.gz 2024-05-08T06:38:15.350 INFO:teuthology.orchestra.run.smithi163.stderr: 2024-05-08T06:38:15.350 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46715.log: /var/log/ceph/ceph-client.admin.48220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49108.log 2024-05-08T06:38:15.350 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46715.log.gz/var/log/ceph/ceph-client.admin.48263.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48220.log.gz 2024-05-08T06:38:15.350 INFO:teuthology.orchestra.run.smithi163.stderr: 2024-05-08T06:38:15.350 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47586.log 2024-05-08T06:38:15.350 INFO:teuthology.orchestra.run.smithi163.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.48263.log.gz 2024-05-08T06:38:15.350 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48774.log 2024-05-08T06:38:15.351 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49108.log: /var/log/ceph/ceph-client.admin.47586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48377.log 2024-05-08T06:38:15.351 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.48774.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49108.log.gz 2024-05-08T06:38:15.351 INFO:teuthology.orchestra.run.smithi163.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47586.log.gz 2024-05-08T06:38:15.351 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49876.log 2024-05-08T06:38:15.351 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48774.log.gz 2024-05-08T06:38:15.351 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42676.log 2024-05-08T06:38:15.351 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.48377.log: /var/log/ceph/ceph-client.admin.49876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45586.log 2024-05-08T06:38:15.351 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.42676.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48377.log.gz 2024-05-08T06:38:15.351 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49876.log.gz 2024-05-08T06:38:15.352 INFO:teuthology.orchestra.run.smithi163.stderr:gzip 87.8% -5 --verbose -- /var/log/ceph/ceph-client.admin.45020.log 2024-05-08T06:38:15.352 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42676.log.gz 2024-05-08T06:38:15.352 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.45586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42670.log 2024-05-08T06:38:15.352 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4%/var/log/ceph/ceph-client.admin.45020.log: -- replaced with /var/log/ceph/ceph-client.admin.45586.log.gz 2024-05-08T06:38:15.352 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47796.log 2024-05-08T06:38:15.352 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45020.log.gz 2024-05-08T06:38:15.352 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41991.log 2024-05-08T06:38:15.352 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.42670.log: /var/log/ceph/ceph-client.admin.47796.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42670.log.gz 2024-05-08T06:38:15.353 INFO:teuthology.orchestra.run.smithi163.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.47796.log.gz 2024-05-08T06:38:15.353 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43755.log 2024-05-08T06:38:15.353 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.41991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43713.log 2024-05-08T06:38:15.353 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43755.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41991.log.gz 2024-05-08T06:38:15.353 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45940.log 2024-05-08T06:38:15.353 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43755.log.gz 2024-05-08T06:38:15.353 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45628.log 2024-05-08T06:38:15.353 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43713.log: /var/log/ceph/ceph-client.admin.45940.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-08T06:38:15.353 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.45628.log: 88.1% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43713.log.gz 2024-05-08T06:38:15.354 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45940.log.gz 2024-05-08T06:38:15.354 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48461.log 2024-05-08T06:38:15.354 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45628.log.gz 2024-05-08T06:38:15.354 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose/var/log/ceph/ceph-mon.a.log: -- /var/log/ceph/ceph-client.admin.43545.log 2024-05-08T06:38:15.354 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.48461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47309.log 2024-05-08T06:38:15.354 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48461.log.gz 2024-05-08T06:38:15.354 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49624.log 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43545.log.gz 2024-05-08T06:38:15.354 INFO:teuthology.orchestra.run.smithi163.stderr: 2024-05-08T06:38:15.355 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47679.log 2024-05-08T06:38:15.355 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49624.log: /var/log/ceph/ceph-client.0.47309.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47309.log.gz 2024-05-08T06:38:15.355 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49540.log 2024-05-08T06:38:15.355 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49624.log.gz 2024-05-08T06:38:15.355 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.47679.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47679.log.gz 2024-05-08T06:38:15.355 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48112.log 2024-05-08T06:38:15.355 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46929.log 2024-05-08T06:38:15.355 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.48112.log: /var/log/ceph/ceph-client.admin.49540.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49540.log.gz 2024-05-08T06:38:15.356 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48112.log.gz 2024-05-08T06:38:15.356 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45712.log 2024-05-08T06:38:15.356 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47491.log 2024-05-08T06:38:15.356 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46929.log: /var/log/ceph/ceph-client.admin.45712.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46929.log.gz 2024-05-08T06:38:15.356 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45712.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42369.log 2024-05-08T06:38:15.356 INFO:teuthology.orchestra.run.smithi163.stderr: 2024-05-08T06:38:15.356 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.47491.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47491.log.gz 2024-05-08T06:38:15.356 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46197.log 2024-05-08T06:38:15.356 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47868.log 2024-05-08T06:38:15.357 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.42369.log: /var/log/ceph/ceph-client.admin.46197.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42369.log.gz 2024-05-08T06:38:15.357 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47313.log 2024-05-08T06:38:15.357 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46197.log.gz 2024-05-08T06:38:15.357 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.47868.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47868.log.gz 2024-05-08T06:38:15.357 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43002.log 2024-05-08T06:38:15.357 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46324.log 2024-05-08T06:38:15.357 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43002.log: /var/log/ceph/ceph-client.0.47313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47313.log.gz 2024-05-08T06:38:15.357 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43839.log 2024-05-08T06:38:15.357 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43002.log.gz 2024-05-08T06:38:15.358 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46324.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46324.log.gz 2024-05-08T06:38:15.358 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43345.log 2024-05-08T06:38:15.358 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41847.log 2024-05-08T06:38:15.358 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43839.log: /var/log/ceph/ceph-client.admin.43345.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43345.log.gz 2024-05-08T06:38:15.358 INFO:teuthology.orchestra.run.smithi163.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43839.log.gz 2024-05-08T06:38:15.358 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41717.log 2024-05-08T06:38:15.358 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42666.log 2024-05-08T06:38:15.358 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.41717.log: /var/log/ceph/ceph-client.admin.41847.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41847.log.gz 2024-05-08T06:38:15.359 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41717.log.gz 2024-05-08T06:38:15.359 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42453.log 2024-05-08T06:38:15.359 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46492.log 2024-05-08T06:38:15.359 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.42666.log: /var/log/ceph/ceph-client.admin.42453.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42666.log.gz 2024-05-08T06:38:15.359 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42453.log.gz 2024-05-08T06:38:15.359 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46887.log 2024-05-08T06:38:15.359 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46492.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46492.log.gz 2024-05-08T06:38:15.359 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44320.log 2024-05-08T06:38:15.360 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41405.log 2024-05-08T06:38:15.360 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46887.log: /var/log/ceph/ceph-client.0.44320.log: 0.0% 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46887.log.gz 2024-05-08T06:38:15.360 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.0.44320.log.gz 2024-05-08T06:38:15.360 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41675.log 2024-05-08T06:38:15.360 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48503.log 2024-05-08T06:38:15.360 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.41405.log: /var/log/ceph/ceph-client.admin.41675.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41405.log.gz 2024-05-08T06:38:15.360 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41675.log.gz 2024-05-08T06:38:15.360 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49498.log 2024-05-08T06:38:15.360 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.48503.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48503.log.gz 2024-05-08T06:38:15.361 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46239.log 2024-05-08T06:38:15.361 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47013.log 2024-05-08T06:38:15.361 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49498.log: /var/log/ceph/ceph-client.admin.46239.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49498.log.gz 2024-05-08T06:38:15.361 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46239.log.gz 2024-05-08T06:38:15.361 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45104.log 2024-05-08T06:38:15.361 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.47013.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47013.log.gz 2024-05-08T06:38:15.361 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29272.log 2024-05-08T06:38:15.361 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48876.log 2024-05-08T06:38:15.361 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.45104.log: /var/log/ceph/ceph.tmp-client.admin.29272.log: 0.0% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45104.log.gz 2024-05-08T06:38:15.362 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph.tmp-client.admin.29272.log.gz 2024-05-08T06:38:15.362 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49066.log 2024-05-08T06:38:15.362 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48732.log 2024-05-08T06:38:15.362 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.48876.log: /var/log/ceph/ceph-client.admin.49066.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48876.log.gz 2024-05-08T06:38:15.362 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41803.log 2024-05-08T06:38:15.362 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49066.log.gz 2024-05-08T06:38:15.362 INFO:teuthology.orchestra.run.smithi163.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-08T06:38:15.362 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.48732.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48732.log.gz 2024-05-08T06:38:15.363 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43349.log 2024-05-08T06:38:15.363 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44315.log 2024-05-08T06:38:15.363 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.41803.log: /var/log/ceph/ceph-client.admin.43349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45062.log 2024-05-08T06:38:15.363 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.44315.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41803.log.gz 2024-05-08T06:38:15.363 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43349.log.gz 2024-05-08T06:38:15.363 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44315.log.gz 2024-05-08T06:38:15.363 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42285.log 2024-05-08T06:38:15.363 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49750.log 2024-05-08T06:38:15.363 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.45062.log: /var/log/ceph/ceph-client.admin.42285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44761.log 2024-05-08T06:38:15.364 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49750.log: 88.4% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45062.log.gz 2024-05-08T06:38:15.364 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42285.log.gz 2024-05-08T06:38:15.364 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49750.log.gz 2024-05-08T06:38:15.364 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45146.log 2024-05-08T06:38:15.364 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48587.log 2024-05-08T06:38:15.364 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.2.44761.log: /var/log/ceph/ceph-client.admin.45146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46803.log 2024-05-08T06:38:15.364 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44761.log.gz 2024-05-08T06:38:15.364 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.48587.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44894.log 2024-05-08T06:38:15.365 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45146.log.gz 2024-05-08T06:38:15.365 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48587.log.gz 2024-05-08T06:38:15.365 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45299.log 2024-05-08T06:38:15.365 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46803.log: /var/log/ceph/ceph-client.admin.44894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43343.log 2024-05-08T06:38:15.365 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.45299.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46803.log.gz 2024-05-08T06:38:15.365 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44894.log.gz 2024-05-08T06:38:15.365 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46450.log 2024-05-08T06:38:15.365 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45299.log.gz 2024-05-08T06:38:15.365 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49792.log 2024-05-08T06:38:15.366 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43343.log: gzip/var/log/ceph/ceph-client.admin.46450.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42672.log 2024-05-08T06:38:15.366 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.49792.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46450.log.gz 2024-05-08T06:38:15.366 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43343.log.gz 2024-05-08T06:38:15.366 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49792.log.gz 2024-05-08T06:38:15.366 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46673.log 2024-05-08T06:38:15.366 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.42672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47549.log 2024-05-08T06:38:15.366 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42672.log.gz 2024-05-08T06:38:15.366 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42243.log 2024-05-08T06:38:15.367 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46673.log.gz 2024-05-08T06:38:15.367 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45188.log 2024-05-08T06:38:15.367 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.42243.log: /var/log/ceph/ceph-client.admin.47549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42495.log 2024-05-08T06:38:15.367 INFO:teuthology.orchestra.run.smithi163.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42243.log.gz 2024-05-08T06:38:15.367 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.45188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48834.log 2024-05-08T06:38:15.367 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% 90.0% -- replaced with /var/log/ceph/ceph-client.admin.47549.log.gz 2024-05-08T06:38:15.367 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45188.log.gz 2024-05-08T06:38:15.367 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.42495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43346.log 2024-05-08T06:38:15.368 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42495.log.gz 2024-05-08T06:38:15.368 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41497.log 2024-05-08T06:38:15.368 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.48834.log: /var/log/ceph/ceph-client.admin.43346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42117.log 2024-05-08T06:38:15.368 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.41497.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48834.log.gz 2024-05-08T06:38:15.368 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43346.log.gz 2024-05-08T06:38:15.368 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41497.log.gz 2024-05-08T06:38:15.368 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43011.log 2024-05-08T06:38:15.368 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43347.log 2024-05-08T06:38:15.368 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.42117.log: /var/log/ceph/ceph-client.admin.43011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43629.log 2024-05-08T06:38:15.369 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43347.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42117.log.gz 2024-05-08T06:38:15.369 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43011.log.gz 2024-05-08T06:38:15.369 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46536.log 2024-05-08T06:38:15.369 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43347.log.gz 2024-05-08T06:38:15.369 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-08T06:38:15.369 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.43629.log: /var/log/ceph/ceph-client.admin.46536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46761.log 2024-05-08T06:38:15.369 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mds.a.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46536.log.gz 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43629.log.gz 2024-05-08T06:38:15.369 INFO:teuthology.orchestra.run.smithi163.stderr: 2024-05-08T06:38:15.369 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35314.log 2024-05-08T06:38:15.370 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48419.log 2024-05-08T06:38:15.370 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46761.log: /var/log/ceph/ceph-client.admin.35314.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46761.log.gz 2024-05-08T06:38:15.370 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35314.log.gz 2024-05-08T06:38:15.370 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42669.log 2024-05-08T06:38:15.370 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42075.log 2024-05-08T06:38:15.370 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.42669.log: /var/log/ceph/ceph-client.admin.48419.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48419.log.gz 2024-05-08T06:38:15.370 INFO:teuthology.orchestra.run.smithi163.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42669.log.gz 2024-05-08T06:38:15.370 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43341.log 2024-05-08T06:38:15.371 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46631.log 2024-05-08T06:38:15.371 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.42075.log: /var/log/ceph/ceph-client.admin.43341.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42075.log.gz 2024-05-08T06:38:15.371 INFO:teuthology.orchestra.run.smithi163.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43341.log.gz 2024-05-08T06:38:15.371 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49960.log 2024-05-08T06:38:15.371 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42998.log 2024-05-08T06:38:15.371 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.46631.log: /var/log/ceph/ceph-client.admin.49960.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46631.log.gz 2024-05-08T06:38:15.371 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49960.log.gz 2024-05-08T06:38:15.371 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47121.log 2024-05-08T06:38:15.372 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48629.log 2024-05-08T06:38:15.372 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.47121.log: /var/log/ceph/ceph-client.admin.42998.log: 88.6% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42998.log.gz 2024-05-08T06:38:15.372 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47121.log.gz 2024-05-08T06:38:15.372 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48066.log 2024-05-08T06:38:15.372 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42677.log 2024-05-08T06:38:15.372 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.48629.log: /var/log/ceph/ceph-client.admin.48066.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48629.log.gz 2024-05-08T06:38:15.372 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42159.log 2024-05-08T06:38:15.372 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48066.log.gz 2024-05-08T06:38:15.372 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45377.log 2024-05-08T06:38:15.373 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.42677.log: /var/log/ceph/ceph-client.admin.42159.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42677.log.gz 2024-05-08T06:38:15.373 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41949.log 2024-05-08T06:38:15.373 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42159.log.gz 2024-05-08T06:38:15.373 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45467.log 2024-05-08T06:38:15.373 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.45377.log: /var/log/ceph/ceph-client.admin.41949.log: 88.3% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45377.log.gz 2024-05-08T06:38:15.373 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41949.log.gz 2024-05-08T06:38:15.373 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.45467.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45467.log.gz 2024-05-08T06:38:15.374 INFO:teuthology.orchestra.run.smithi192.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-08T06:38:15.375 INFO:teuthology.orchestra.run.smithi192.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-08T06:38:15.375 INFO:teuthology.orchestra.run.smithi192.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-08T06:38:15.375 INFO:teuthology.orchestra.run.smithi163.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-08T06:38:15.375 INFO:teuthology.orchestra.run.smithi163.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-08T06:38:15.384 INFO:teuthology.orchestra.run.smithi163.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-08T06:38:15.389 INFO:teuthology.orchestra.run.smithi163.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-08T06:38:15.391 INFO:teuthology.orchestra.run.smithi192.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-08T06:38:15.402 INFO:teuthology.orchestra.run.smithi163.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-08T06:38:15.415 INFO:teuthology.orchestra.run.smithi192.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-08T06:38:15.429 INFO:teuthology.orchestra.run.smithi192.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-08T06:38:15.431 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-05-08T06:38:15.431 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m0.170s 2024-05-08T06:38:15.431 INFO:teuthology.orchestra.run.smithi192.stderr:user 0m0.700s 2024-05-08T06:38:15.431 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m0.048s 2024-05-08T06:38:15.443 INFO:teuthology.orchestra.run.smithi163.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-08T06:38:15.445 INFO:teuthology.orchestra.run.smithi163.stderr: 2024-05-08T06:38:15.445 INFO:teuthology.orchestra.run.smithi163.stderr:real 0m0.209s 2024-05-08T06:38:15.445 INFO:teuthology.orchestra.run.smithi163.stderr:user 0m0.790s 2024-05-08T06:38:15.446 INFO:teuthology.orchestra.run.smithi163.stderr:sys 0m0.146s 2024-05-08T06:38:15.446 INFO:tasks.ceph:Archiving logs... 2024-05-08T06:38:15.446 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/log/ceph to /home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389/remote/smithi163/log 2024-05-08T06:38:15.447 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T06:38:15.739 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389/remote/smithi192/log 2024-05-08T06:38:15.740 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T06:38:15.877 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T06:38:15.896 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T06:38:15.896 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T06:38:15.899 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T06:38:15.973 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T06:38:15.983 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T06:38:15.983 DEBUG:teuthology.orchestra.run.smithi163:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T06:38:15.986 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T06:38:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-05-08T06:38:16.000 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-05-08T06:38:16.002 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T06:38:16.002 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-05-08T06:38:16.002 INFO:teuthology.orchestra.run.smithi163.stdout:^* hv01.front.sepia.ceph.com 3 6 177 63 +543us[ +506us] +/- 54ms 2024-05-08T06:38:16.003 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv02.front.sepia.ceph.com 3 6 177 62 -1524us[-1524us] +/- 64ms 2024-05-08T06:38:16.003 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv03.front.sepia.ceph.com 3 6 177 62 +1001us[+1001us] +/- 74ms 2024-05-08T06:38:16.003 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:38:16.005 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T06:38:16.005 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-05-08T06:38:16.005 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv01.front.sepia.ceph.com 3 6 177 62 +562us[ +576us] +/- 54ms 2024-05-08T06:38:16.005 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv02.front.sepia.ceph.com 3 6 177 63 -1548us[-1533us] +/- 64ms 2024-05-08T06:38:16.005 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 177 62 +991us[+1005us] +/- 74ms 2024-05-08T06:38:16.005 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:38:16.006 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T06:38:16.023 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T06:38:16.025 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T06:38:16.051 DEBUG:teuthology.orchestra.run.smithi163:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T06:38:16.116 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T06:38:16.177 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T06:38:16.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:38:16.205 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T06:38:16.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:38:16.235 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T06:38:16.246 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T06:38:16.256 INFO:teuthology.task.internal:Duration was 503.452484 seconds 2024-05-08T06:38:16.256 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T06:38:16.266 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T06:38:16.266 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T06:38:16.268 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T06:38:16.305 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T06:38:16.313 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T06:38:16.681 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T06:38:16.681 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:38:16.682 DEBUG:teuthology.orchestra.run.smithi163:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T06:38:16.741 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T06:38:16.742 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T06:38:16.772 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T06:38:16.772 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T06:38:16.784 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T06:38:16.838 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T06:38:16.838 DEBUG:teuthology.orchestra.run.smithi163:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T06:38:16.850 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T06:38:16.976 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T06:38:16.992 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T06:38:16.992 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T06:38:17.018 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T06:38:17.043 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T06:38:17.054 DEBUG:teuthology.orchestra.run.smithi163:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:38:17.062 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:38:17.083 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = core 2024-05-08T06:38:17.106 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-05-08T06:38:17.119 DEBUG:teuthology.orchestra.run.smithi163:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:38:17.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:38:17.151 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:38:17.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:38:17.173 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T06:38:17.185 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T06:38:17.186 DEBUG:teuthology.misc:Transferring archived files from smithi163:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389/remote/smithi163 2024-05-08T06:38:17.186 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T06:38:17.226 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389/remote/smithi192 2024-05-08T06:38:17.226 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T06:38:17.262 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T06:38:17.262 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T06:38:17.266 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T06:38:17.310 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T06:38:17.323 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T06:38:17.323 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T06:38:17.334 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T06:38:17.334 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T06:38:17.337 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T06:38:17.349 INFO:teuthology.orchestra.run.smithi163.stdout: 265850 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 06:38 /home/ubuntu/cephtest 2024-05-08T06:38:17.365 INFO:teuthology.orchestra.run.smithi192.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 06:38 /home/ubuntu/cephtest 2024-05-08T06:38:17.366 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T06:38:17.376 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T06:38:17.398 INFO:teuthology.nuke:Checking targets against current locks 2024-05-08T06:38:17.426 DEBUG:teuthology.nuke:shortname: smithi163 2024-05-08T06:38:17.427 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T06:38:17.450 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 06:20:10.656817', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEjGUnfpYuz1hfmfIk1olfiv3+NVWGgwOJdsPkuNzIFSejmYiKRr2J8ipCci639i/dE2wiRCQvoWj1zoKeuqzoA='} 2024-05-08T06:38:17.454 DEBUG:teuthology.nuke:shortname: smithi192 2024-05-08T06:38:17.454 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T06:38:17.476 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-05-08_06:15:23-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697389', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 06:20:10.658358', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKPBptJFAXy1RPfSH9ism4orrVF6u8zHozwHRD2UWZFoURNJg4XfWnLMB1IkroVAcJHKNX/FdJ7TvKDtZv80fsk='} 2024-05-08T06:38:17.506 INFO:teuthology.orchestra.console.smithi163:Power off 2024-05-08T06:38:17.507 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-08T06:38:17.534 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off 2024-05-08T06:38:17.534 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:38:17.557 INFO:teuthology.orchestra.console.smithi192:Power off 2024-05-08T06:38:17.558 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-08T06:38:17.568 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-05-08T06:38:17.581 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2024-05-08T06:38:17.581 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:38:17.604 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-05-08T06:38:21.569 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:38:21.606 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:38:21.681 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-05-08T06:38:21.716 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-05-08T06:38:25.686 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:38:25.717 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:38:25.798 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-05-08T06:38:25.826 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-05-08T06:38:29.801 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:38:29.827 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:38:29.912 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off 2024-05-08T06:38:29.937 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2024-05-08T06:38:30.014 INFO:teuthology.orchestra.console.smithi163:Power off completed 2024-05-08T06:38:30.038 INFO:teuthology.orchestra.console.smithi192:Power off completed 2024-05-08T06:38:30.144 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/mds-flush} duration: 503.4524836540222 failure_reason: 'Test failure: test_flush (tasks.cephfs.test_flush.TestFlush)' flavor: default owner: scheduled_jcollin@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=ccf80d23be9d4f99af39ad8396cdb936 status: fail success: false 2024-05-08T06:38:30.145 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T06:38:30.429 INFO:teuthology.run:FAIL